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WHAT WORKS? 



COMMON PRACTICES IN HIGH FUNCTIONING AFTERSCHOOL PROGRAMS 
ACROSS THE NATION IN MATH, READING, SCIENCE, ARTS, TECHNOLOGY, 
AND HOMEWORK— A STUDY BY THE NATIONAL PARTNERSHIP 



Denise Huang, Jamie Cho, Sima Mostafavi, and Hannah H. Nam 
CRESST/University of California, Los Angeles 

ABSTRACT 

In an effort to identify and incorporate exemplary practices into existing and future 
afterschool programs, the U.S. Department of Education commissioned a large-scale 
evaluation of the 21st Century Community Learning Center (CCLC) program. The 
purpose of this evaluation project was to develop resources and professional development 
that addresses issues relating to the establishment and sustainability of afterschool 
programs. Fifty-three high functioning programs representative across eight regional 
divisions of the nation, including rural and urban programs, community-based and school 
district related programs, were identified using rigorous methods. Exemplary practices in 
program organization, program structure, and especially in content delivery were studied. 

The findings were synthesized into the Afterschool Toolkit that was made available to 
programs nationwide via the world-wide-web. Professional development was conducted 
consistently and extensively throughout the nation. 

INTRODUCTION 

Since the implementation of the No Child Left Behind Act (NCLB, 2002), a greater 
emphasis has been placed on academic development during the afterschool hours. Research 
has found that students’ participation in afterschool programs is beneficial to academic 
achievement and social adjustment (Pierce, Hamm, & Vandell, 1999; Posner & Vandell, 
1994). In fact, a recent longitudinal study conducted by Mahoney, Lord, and Carryl (2005) 
found that students who participated in afterschool programs had significantly higher reading 
achievement and were rated by teachers as having a greater expectancy of success than 
students who did not participate in afterschool programs. 

One mechanism in which afterschool programs influence students’ academic 
achievement is through the provision of homework assistance. An evaluation of 21st Century 
Community Learning Center (CCLC) programs conducted by Mathematica Policy Research 
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(Dynarski, et al., 2003) noted that homework sessions tend to be the most common type of 
academic support both in elementary and middle school programs; yet the quality of support 
is generally low. At the same time, studies have found that homework and tutoring programs 
that rely on untrained or minimally trained volunteers often do little to boost students’ 
academic performance (Fashola, 2002). Instead, research on effective approaches to teaching 
and learning continue to emphasize the need to actively engage students and delve deeply 
into subject matter, providing “opportunities to leam with understanding” in combination 
with “a deep foundation of factual knowledge” (Bransford, Brown, & Cocking, 2000, p. 16). 

In addition to the challenges around hiring qualified staff members in afterschool 
programs, afterschool programs can also be plagued with attrition and a low frequency of 
student participation (Dynarski, et al., 2003; Grossman, et al., 2002). Attrition and low 
attendance are of particular concern because research has found that those students whom 
participate in afterschool programs the longest (both in terms of frequencies and durations) 
make the biggest gains (Goldschmidt, Huang, & Chinen, 2007; Huang et. al, 2006; Lamare, 
1997) and higher rates of participation in afterschool programs resulted in higher scores on 
academic standardized tests in mathematics, reading, and language arts (Huang, Gribbons, 
Kim, Lee, & Baker, 2000). Similarly, Munoz (2002) found a positive correlation between 
number of visits to afterschool programs and improved school attendance and academic 
achievement. Thus students who infrequently attend the afterschool programs are unlikely to 
reap academic and social benefits. 

In an effort to identify and incorporate exemplary practices into existing and future 
afterschool programs, the U.S. Department of Education decided to commission a large-scale 
evaluation of the 21st CCLC program. The purpose of this evaluation project was to: develop 
resources and professional development that addresses issues relating to the establishment 
and sustainability of afterschool programs; provide models, indicators of promising practices, 
and other descriptive information that local sites can access in planning new afterschool 
programs or improving existing ones (including the Promising Practices in Afterschool 
System and the Harvard Family Research Project’s database of afterschool programs); and 
assess the effectiveness of afterschool programs in general, including attention to 
performance standards, review and meta-analyses of research, and rigorous evaluation 
methods to identify “what works.” 

In 2003, through a competitive solicitation process, the National Partnership for Quality 
Afterschool Learning, consisting of the Southwest Educational Laboratory (SEDL) and in 
partnership with the National Center for Research on Evaluation, Standards, and Student 
Testing (CRESST), the Mid-Continent Resources for Education and Learning (McREL), the 
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Northwest Regional Educational Laboratory (NWREL), and the WGBH Educational 
Foundation, SERVE Inc., and the Institute for Responsive Education (IRE) 1 , contracted with 
the U.S. Department of Education in this project to support program quality among the 
growing number of 21st CCLCs throughout the United States. This 5-year project provided 
strategies, tools, and technical assistance to afterschool programs to address two continuing 
challenges identified via research on afterschool programs: (a) Eensure that programs offer 
high quality, researched-based academic content utilizing appropriate methods of teaching 
and learning, and (b) Eensure that programs are able to attract and retain students who 
participate regularly and thus can benefit from these investments. 

There were five major tasks for this study aiming at improving the delivery and quality 
of academic content, teaching, and professional development in afterschool programs. 

Task 1. Identification of afterschool sites across the U.S. that are demonstrating 
exemplary or promising practices. 

Task 2. Validation of afterschool success in the content areas of reading, math, 
science, arts, technology, and homework help through data analysis and site visitation. 

Task 3. Product development of tools, models, expertise, and other assistance to 
increase the number of promising and exemplary afterschool sites across the U.S. 

Task 4. Provision of technical assistance to promising afterschool sites to help them 
achieve “exemplary” status, as well as support of state education agencies in building 
their own capacity for technical assistance in order to, in turn, assist grantees in the 
same technical assistance-building process. 

Task 5. Partnering with the U.S. Department of Education and state education 
agencies to provide professional development opportunities for afterschool sites in 
adopting promising and exemplary practices, specifically in content areas designed to 
increase student achievement and attract high levels of student participation. 

The role of CRESST within the National Partnership for Quality Afterschool Learning 
(NPQAL) was to accomplish Tasks 1 and 2 and to advise on the implementation of Tasks 
3-5 by SEDL. In Years 1-3 of the study, Task 1 was completed after a comprehensive search 
and selection procedures involving primary and secondary screenings of afterschool 
programs throughout the U.S. These screenings were based on an established set of criteria, 
including evidence of academic performance of attendees, Annual Performance Report 
(APR) data, 21st CCLC Profile and Performance Information Collection System (PPICS) 
data, sample size, and recommendations from regional partners. As a result of this process, a 
total of 53 afterschool programs were identified and recruited for site visits by Partnership 
researchers. For Task 2, site visits were conducted at two sites under each afterschool 



1 The IRE left the NPQAL in 2004. 
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program, and interviews were conducted with the project director, site coordinators, and 
program instructors. In addition, surveys were administered to site instructors, site 
coordinators, and parents. For Task 3, toolkits and self-assessments were created, based on 
this study’s results, to provide afterschool programs with research-based tools and strategies 
to improve curriculum implementation in the six content areas (math, reading, science, 
technology, arts, and homework), and to evaluate their programs regularly and 
independently. 

Purpose of the Report 

This report provides a synthesis of the qualitative and quantitative findings for the field 
work conducted across the nation and the validation of the promising practices across the six 
content areas, triangulating interviews, survey, and observation data. The following sections 
describe the 13 criteria presented in CRESST’s Theoretical Logic Model (see Figure 1) and 
the study design, methods, and results of the validation of promising and exemplary practices 
in these content focuses: 

1. The Study Design Section discusses the program selection process, as well as 

CRESST’s process in developing the indicator system that guided validation. 

2. The Methodology and Validation Procedures Section explains the Partnership’s data 

collection processes, and elaborates on evaluation methodology, procedures, and 
instruments employed by CRESST in the analyses. 

3. The Internal Program Structure Section describes the Internal Program Structure of 

the 53 afterschool programs in the study sample. 

4. Program Process Section provides the findings on program process, including parent 

involvement, connecting with the community, and relationship building with the 
students. 

5. Goal Setting and Curricular Practices Section examines program-based content 

practices in terms of curricular goals, alignment with state standards, and links to 
day-school curricula. This section also summarizes findings on student enrichment 
and research-based practices. 

6. The Evaluative Structures and Program Impact Section summarizes the results of 

program impact, and 

7. Implications and Conclusion Section highlights overall common exemplary 

afterschool practices, as well as the quantitative analysis. 

STUDY DESIGN 

Identification of Promising Afterschool Practices 

This section describes the Partnership’s data collection processes, and elaborates on 
methodology, procedures, and instruments developed by CRESST. The first year of the study 
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focused on reading and math content practices, the second year of the study focused on 
science and arts content practices, and the third year of the study focused on technology and 
homework help practices. First, an indicator system was designed to guide the instrument 
development and validation procedures. 

Preparation Process for Development of the Indicator System and Standards 

To develop an indicator system for program identification and validation, CRESST 
drew on a number of sources for information and expertise throughout the process. First was 
an extensive review of the existing research literature on afterschool programs, investigating 
the organizational, curricular, and environmental variables that have been linked to program 
quality. CRESST then reviewed publications from organizations across the country involved 
in afterschool program evaluation and support. These organizations included the National 
Center for Community Education, Promising Practices in After School Systems, the National 
Institute on Out-of-School Time, the Harvard Family Research Project, the National 
Community Education Association, the After School Alliance, the After School Corporation, 
the C.S. Mott Foundation, Learning Point Associates, Manpower Demonstration Research 
Corporation, and the Institute for Educational Science. A review of these sources focused on 
common variables and processes associated with positive afterschool program outcomes. 

CRESST also received considerable guidance for indicator development from the 
Partnership’s Content Focus Teams. Math is lead by McREL, Reading is lead by NWREL, 
Science and Homework Help are led by SERVE, and Arts and Technology are led by SEDL. 
Harris Cooper, the Steering Committee member who is the recognized expert in Homework 
studies provided his expert guidance in homework and technical issues, as did Liz Reisner, 
Priscilla Little, and other members of the Steering Committee. Teams convened and provided 
CRESST with their feedback on key curriculum content in their respective areas of expertise. 

The Indicator System to Guide the Validation Process 

To design an indicator system that could validate promising and exemplary practices, it 
was necessary to define the essential elements in a quality afterschool program. In general, 
the quality of afterschool programs was defined as the ability “to deliver basic developmental 
inputs, which translate into practices and principles at the staff, program, and organizational 
level, and which result in positive outcomes for participants” (Hall, Yohalem, Tolman, & 
Wilson, 2003, p.51). Child development theories such as those by Bronfenbrenner (1979), 
Confrey (1991), Piaget (1952, 1964), and Vygotsky (1978) were helpful in providing a better 
understanding of how afterschool programs benefit student learning. These theories describe 
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the importance of the social context of learning and how learning or cognitive development 
can be stimulated through interaction with peers and adults. 

After extensive literature review and consultation, three central themes emerged: 
program quality evolves from the (a) setting of goals and evaluations, (b) curricular quality, 
and (c) program environment. Setting of goals and evaluations involves theory and research- 
based programming, program structure, setting of desired outcomes, and ongoing evaluation 
to ensure that goals are being met. Determination of curricular quality entails academic 
collaboration (linkage of afterschool programs to the day school), and opportunities for 
students to practice skills and reinforce motivation and engagement. Understanding of the 
program environment includes social collaboration (school-staff communication), adequacy 
of physical and human resources. A logic model (Figure 1) was developed to guide the 
validation process of identifying promising and exemplary practices. 




Figure 1. Theoretical logic model. 
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This model recognized 13 components indicative of success in delivering quality 
content and maintaining effective functionality in an afterschool setting: 

• Setting of clear goals for content area practice (i.e., reading, math, science, arts, 
technology, and homework help) 

• Alignment of research-based activities to achieve goals 

• Alignment of practice content materials with state standards 

• Links between practice content activities and day-school activities 

• Use of research-based curriculum and teaching strategies 

• Provision of a positive program environment 

• Employment of motivational strategies to engage students in learning 

• Promotion of student engagement (e.g., encouraging meaningful experiences) 

• Effective program management/support/resources (e.g., staff/student ratio, staff 
educational experience, ongoing evaluation) 

• Provision of opportunities for student practice 

• Periodic evaluation to check program effectiveness 

• Periodic assessment to review student progress 

• Resetting goals according to assessment results 

These indicators guided CRESST’s development of observation and site visit protocols, 
and aided partnership researchers in the implementation of the resulting instruments. 
Furthermore, this model provides a framework in examining the larger context of the 
afterschool environment, and assessing the effectiveness and efficiency of content practices 
at participating sites. One of the main objectives of this final report is to highlight the 
components of the theoretical logic model that are most likely to lead to program success. 

Program Identification 

The first task on this project was to identify 21st CCLC programs across the U.S. that 
are demonstrating exemplary or promising practices in six content areas: reading, math, 
science, arts, technology, and homework help. Programs in all the content areas were 
selected based upon nominations from state coordinators and experts within the field, 
APR/PPICS data, and telephone interviews with program representatives. The next section 
describes the process by which the programs were selected for the study sample. 
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Program Criteria 

The APR/PPICS were the primary data sources used to identify promising programs.' 
These reports were submitted by the 21st CCLC programs, and were provided to CRESST by 
the U.S. Department of Education. The APRs provided information including program 
objectives, grade levels served, number of students served, student demographics, student 
academic achievement data, hours/days per week the specific content curriculum offered, 
number of staff members in the program, and percentage of credentialed staff members. 
Information from 1,600 grantees was available for the selection process; however, due to the 
variation in program types and size with different curricula foci, criteria for selection were 
different across the different content areas. The criteria for each year are listed in the 
following text: 

Year 1 - Reading and Math: 

• Each program should have 100 students or more. 

• Each program should have a minimum of 3 years of operation history to ensure that the 

content practice has the capacity to maintain its effects and show some signs of stability. 

• Each program should show that at least three of its sites offer the requisite content 

practices, as an indicator of stability and to improve the probability for the future success 

of practice duplication. 

• Content practices should be offered a minimum of three times per week in order to allow 
sufficient time for student exposure to practices and materials, and to strengthen dosage 
effects. 

Year 2 - Science and Arts: 

• Each program should have 50 students or more. 

• Each program must show a minimum of 3 years of operation history to ensure that the 

content practice has the capacity to maintain its effects and show some signs of stability. 

• Each program must show that at least two of its sites are offering the requisite content 
practices, as an indicator of stability and to improve the probability for the future success 
of practice duplication. 

• Content practices must be offered a minimum of two times per week in order to allow 
sufficient time for students to be exposed to the practices and to absorb the materials. 

• Content practice at both arts and science programs should have a strong focus on at least 
one of the major disciplines. For arts programs, this could include a focus on dance, 
music, theater, or visual arts. For science programs, this could include a focus on physical 
science, life science, earth and space science, science and technology, science in personal 
and social perspectives, or history and nature of science. 



2 

' However, for Year 2, the U.S. Department of Education contracted Learning Point to convert the APR into 
electronic versions called PPICS. 
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Year 3 - Technology and Homework: 

• Each program should have 50 students or more. 

• Each program must show a minimum of 2 years of operation history to ensure that the 
content practice has the capacity to maintain its effects and show some signs of stability. 

• Each program must show that at least two of its sites are offering the requisite content 
practices, as an indicator of stability and to improve the probability for the future success 
of practice duplication. 

• For Technology Only: Each program must show a minimum of 1 year using the 
technology curriculum. 

Most important was the evidence of success in promoting student learning, as derived 
from the 2002 APRs for Year 1, 2004 APRs and PPICS for Year 2, and 2005 PPICS for Year 
3 in the following categories: 

1. Meeting/exceeding goals (as stated it the APR/PPICS). Programs that exceeded any 
of their stated goals were identified. 

2. Teacher survey results. The 10 items constituting the teacher surveys reported in the 
APR/PPICS were reduced to one dimension using factor analysis (each of the items 
correlated with the single factor with correlations ranging from .71 to .92). Higher 
scores indicated more “yes” responses to the teacher survey questions, generally 
implying higher quality. 

3. Academic performance of attendees. Reported academic data came in three forms: 
(a) grade gains, (b) grade levels (above proficiency, at proficiency, etc.), and (c) 
percentile ranks. Because only a small subset of programs reported both percentile 
ranks and grade level gains scores for their attendees, it was necessary to 
standardize each score (mean = 0, standard deviation = 1) to make the different 
metrics reasonably comparable. To assure that programs working with low 
academic performers would not be penalized, a matched gains procedure was used. 
This process allowed programs starting with very low achievement, but 
demonstrating significant gains, an equal opportunity to be recognized and selected. 
Thus, academic scores from all program sites were rescaled and measures of 
percentage gains over the previous academic year were created. These standardized 
scores were then aggregated from the site to the program level and weighted by the 
number of attendees at each site. 

Next, these five sets of scores (exceeding performance, teacher survey results, grade 
change, grade level change, and/or percentile rank change) were correlated to determine the 
consistency of results. The correlations were surprisingly low across the measures, and this 
necessitated focusing on the top performers by content area (i.e., basing standardized units 
above the mean grade level change or percentile rank change). In this way, even though the 
units of measure were not exactly the same, it was less significant because it was a center’s 
deviation from the mean that was most critical. 
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Finally, although the original intention was to select only those afterschool programs 
demonstrating high achievement gains and exceeding their goals, relatively few of the top 
performers exceeded their goals. Thus the original criteria were modified to require that at 
minimum, selected afterschool programs should meet all of their stated goals. Furthermore, 
given that academic scores were unlikely to capture the totality of program quality, steps 
were taken to expand the pool beyond those showing the greatest academic gains. All 
programs exceeding one or more of their goals were also included in the final sample, as long 
as they did not demonstrate any academic declines. In Year 1, as a blinded procedure to 
validate the site selection methodology, five math and five reading programs were randomly 
selected and blended into the final pool to serve as comparison control. The end result was 
the selection of 53 programs including 15 for math, 15 for reading, 9 for science, 9 for arts, 9 
for technology, and 10 for homework. Furthermore, in order to assure promising practices in 
the general population are not overlooked, more than seven non-CCLC programs^ were also 
recommended by State Region Coordinators and experts in the field to be included for further 
investigation. 

Secondary Screening 

After the initial selection of 342 programs (i.e., 47 reading and math programs, 157 
science and arts programs, and 138 technology and homework help programs), additional 
screening activities occurred to narrow the pool of 342 down to 10 promising program 
candidates in each content area: 

Telephone Screening 

CRESST staff and the NPQAL staff (blinded to the control procedures) attempted 
contact with each of the selected programs by telephone in order to introduce them to the 
project and to collect updated and/or additional program information that was not available 
through the APR/PPICS. The telephone screening with program representatives covered 
three primary topics: (a) program background, (b) content focus, and (c) self- 
evaluation/assessment methods. In terms of basic background information, the 
representatives were asked how long the site had been in operation, how many sites existed 
within their program, the population(s) served, the number of days and/or hours a week the 
program was in operation, and how they would characterize the relationship, if any, between 
the afterschool program and the day school. Representatives were also asked about program 
goals, content areas covered, curricular links to standards, the nature of program instruction 
(purchased versus self-designed curricula, types of activities provided in the instruction of 

3 Programs not funded by the 21st Century Community Learning Centers. 
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the six content areas, and components of their programs that were particularly noteworthy. 
Screening questions also focused on internal and external program evaluations and evidence 
of specific content impact on student achievement, learning, retention, and attendance. 

Representatives were also asked to provide supporting materials including: program 
brochures, sample lesson plans, goals and objectives, curricular mapping documents (i.e., 
linking afterschool content area instruction with state standards or day-school curriculum), 
and any other evidence of effective practice that the programs were willing to share. In some 
cases, CRESST used program evaluation reports in the selection process. Programs were also 
asked to recommend two of their sites for the ‘Best Practices’ site visits, in the event that 
their program was selected as part of the final sample. For those CCLC programs that were 
selected, the two sites identified by APR/PPICS data were discussed with the Project 
Director for a final decision. 

Through the secondary screening process the number of programs was narrowed and 
those meeting the criteria were submitted for a final approval process. It should also be noted 
here that the 10 random programs thrown into the selection pool as control measures were 
screened out during the process. Thus validating the screening procedure was effective. 

Final Sample Selection 

The proposed list of pre-identified programs was reviewed by CRESST researchers and 
sent to all partners and steering committee members within NPQAL for review and 
suggestions. In addition to the preliminary criteria previously mentioned, program finalists 
were selected based on factors such as: standards and research-based curricula and 
instruction, links to day school, effective integration of content practices into afterschool 
program instruction, staff qualifications, and evidence of program impact. After careful 
screening and group deliberations, 60 program finalists 4 were selected and presented to our 
Content Experts, the Steering Committee, and the U.S. Department of Education for 
suggestions and feedback. The NPQAL, the Steering Committee, and the U.S. Department of 
Education reviewed and approved the list, and the NPQAL staff contacted the approved 
afterschool programs to request their participation in the project. In total, 53 programs 5 
agreed to participate in the site visits. 



10 candidates from each content area. 

5 The program sample consisted of 11 reading, 7 math, 9 arts, 9 science, 10 technology, and 7 homework help 
programs. The number of reading programs was increased to 11 as one program that was initially chosen as part 
of the math sample was found to be more focused on reading during the site visits. 
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As detailed in the Methodology section of this report, two sites within each of the 
selected afterschool programs were included in the site visits. Specific considerations were 
taken to ensure that rural programs were represented in the final selections. Table 1 shows 
the geographical distribution of the 53 final programs selected. As shown in the table, there 
are eight rural programs and 45 urban programs. The representation is highest at the Mid- 
South region with 10 programs selected, followed by South-West and North-East with nine 
programs selected in each region. 

Table 1 

Geographical Distribution of Afterschool Programs 



Regions: Rural (R) /Urban (U) 



Content 

Areas 


North- 

West 


South- 

West 


Mid- 

South 


Mid- 

West 


South- 

East 


Mid- 

Atlantic 


North- 

East 


North- 

Central 


Total 


R U 


R U 


R U 


R U 


R U 


R U 


R U 


R U 


Reading 


1 


2 


1 


1 


1 


1 


2 


2 


11 


Math 




1 


2 


1 


1 


1 


1 




7 


Art 


2 


2 


1 


1 




1 


1 


1 


9 


Science 




1 


1 3 


1 


1 


1 


1 




9 


Technology 




3 


1 1 


2 


1 




2 




10 


Homework 


2 








1 


1 


2 


1 


7 


Total 


0 5 


0 9 


2 8 


2 4 


1 4 


1 4 


2 7 


0 4 


53 



As seen in Table 2, out of the 53 programs, 33 (62%) are affiliated with school districts, 
and 20 programs are community-based (38%). Within the 20 community-based programs, 10 
of them are run by large non-profit organizations such as YMCA, Boys and Girls club, LA’s 
BEST, Foundations Inc., The After School Corporation, Children’s Aid Society, etc. In 
addition, science, arts, and technology programs each have two programs that are actively 
collaborating with local universities or colleges. Except for science (both of the science 
programs that collaborated with local universities are district-affiliated), the large non-profit 
community-based programs were more likely to collaborate with local universities and 
colleges. Some of the math, reading, and homework programs used local college students as 
assistants and volunteers, but none of them were observed to be actively collaborating with 
universities or colleges. Additionally, none of the programs identified in this study were 
faith-based. 
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Table 2 

Programs Affiliations by Content 





Affiliated with 
school districts 


Community -based 


Content area 


CBO 


Non-Profit 


Reading 


6 


2 


3 


Math 


4 


2 


1 


Art 


6 


2 


1 


Science 


6 


2 


1 


Technology 


6 


1 


3 


Homework 


5 


1 


1 


Total 


33 


10 


10 



Note. CBO = Community-based organization. 

METHODOLOGY AND VALIDATION PROCEDURES 



In order to provide information that can be useful in improving academic content, 
teaching, and professional development in afterschool programs throughout the United 
States, the current study focused on instructional quality by identifying and gathering 
information on programs exhibiting “best practices.” A multi-method strategy incorporating 
both quantitative and qualitative data analyses — including surveys, site observations, and 
interviews — was employed in data collection. SEDL staff and partners conducted site visits 
and data collection on identified programs from 2004-2007. The following sections describe 
the study participants, the measures and protocols used, as well as the data collection 
procedures. 

Participants 

Program Sample 

As described in the Introduction Section of this report, all programs were selected for 
this study using a complex process that included the development of selection criteria, a 
review of APR/PPICS and other data sources, and secondary screenings by telephone 
interview. The final sample consisted of 53 programs located throughout the United States. 
Within each program, two specific sites were selected for visitation. These sites were chosen 
based on recommendations by a program representative who had participated in the 
secondary screening process. 

Most of the programs had been in operation for less than 10 years, with nearly all 
programs reporting between 3-7 years of operation. Depending on the geographical regions 
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of the programs, the populations served across most programs were ethnically diverse, 
serving primarily lower income students. The most common ethnic group served was 
Hispanic or African American, but other populations included Asian/Pacific Islanders, 
Portuguese, Dominican, and Whites. Whereas White students had higher representation in 
the rural programs, most urban programs served a majority of English Language Learners 
populations, with Spanish being the most common primary language. 

Based on the theories and indicators of the theoretical logic model, a more content 
focused validation model (Ligure 2) was designed to provide structural framework for 
instrument development, examination of specific content practices, and data analyses for this 
study. 



Indicators Design & Process Immediate Outcome Expected Outcome 




Figure 2. Validation model. 
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As shown in the validation model, high quality content practices are expected to show 
strength in their curricular content as indicated by linkage with school, linkage to standards, 
and research-based practices. The Internal program structure is indicated by program 
organization, duration and frequency of instruction, staff experience and education, 
govemance/leadership, and professional development. The program process is indicated by 
external communication and support, student/staff interaction, and parent satisfaction. The 
main focus of this study will be on program content, and examining how program process 
can enhance curricular content and student engagement in math, reading, science, arts, 
technology, and homework (in Appendix A content- specified models are shown). 

Measures/Instruments 

All data collection instruments were developed by CRESST. Different versions of the 
instruments were created to cover the six content areas. The following is a brief overview of 
these qualitative measures: 

Staff Survey: The staff survey was administered to participating afterschool program 
site coordinators and instructors. The survey included questions about content- specific 
curriculum, general instructional practices and activities, and organizational or 
structural characteristics. 

Parent Survey: A survey was distributed to parents of the student participants. Items 
focused on perceived program satisfaction, opportunities for parental involvement, as 
well as student impact. 

Teacher Survey: A survey was distributed to day-school teachers whose students were 
also in the afterschool program for Years 2 and 3. 6 Survey questions looked at their 
knowledge of the afterschool curriculum, as well as perceived changes in student 
attitudes, knowledge, and performance due to participation in the program. 

Interview Protocols: The purpose of the interview was to gather information on the 
general nature and structure of the afterschool program to better identify the qualities of 
an exemplary, “best practices” program. Various forms of the interviews were 
developed for project directors, site coordinators and site instructors to specifically 
address questions that were most relevant to the interviewee. Overall, all interview 
protocols covered a number of areas including general program background 
information, content area instructional strategies and other student-based activities, 
program organization and structure, external communication and support, student-adult 
interactions, and program outcomes and impact on students. 

Observation Protocol: A structured observation protocol, including both content- 
specific and general instruction items, was used. The protocol included scales, 
checklists, and open-ended questions focusing on the content and quality of after- 
school instructional practice. The closed-ended portion of the instrument allowed 



6 



Teacher surveys were not utilized for Year 1 due to scheduling conflicts. 
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researchers to assess specific aspects of program implementation and structure in the 
different content areas. The open-ended portion of the observation instrument provided 
an overall description of the lesson and classroom climate. 

Observation Report: The site visit team completed an observation report based on the 
report template provided by CRESST. The information provided in the observation 
reports was used to supplement and triangulate with the other data sources. 

CRESST provided a 2-day professional development for the NPQAL staff and content 
advisory team consultants who conducted the site visits. The first professional development 
occurred in March 2004 at the UCLA campus and additional development was provided via 
teleconference for new site visit staff later that year. For Years 2 and 3, CRESST provided 
training via teleconference only. The professional development covered: 

■ Pre-site visit preparation activities 

■ Understanding of site visit organization and scheduling 

■ Survey collection 

■ Use of interview and observation protocols 

■ Collection of archived data 

■ Debriefing/completing site visit reports 

Procedures: Qualitative Methodologies 

Site Visit Plan 

Two sites within each program were included in the site visit plan. At each site, 
surveys, interviews, and observations were completed. CRESST provided a program-level 
checklist to guide the collection of all data and any additional information shared regarding 
the program. 

Observations 

NPQAL researchers observed two instructors at each site. Site visit teams prepared 
observation reports based on the report template developed by CRESST to provide a 
summary of their site visit experiences. These reports, field notes, and any supplemental 
materials collected were then sent to CRESST and the information provided was used to 
further triangulate the findings. Table 3 shows the total number of observations and number 
of programs and sites visited under each content area. 
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Table 3 

Number of Site Visits and Observations 



Content area 


Total number of 
programs 


Total number of site 
visits 


Total number of 
observations 


Reading 


ii 


20 


38 


Math 


7 


14 


30 


Arts 


9 


16 


48 


Science 


9 


18 


32 


Technology 


10 


14 


31 


Homework 


7 


14 


25 


Total 


53 


44 


104 



Interviews 

Project directors, site coordinators, instructors and other afterschool program staff 
members who volunteered to participate in the study (N = 338) were interviewed in person 
by NPQAL researchers. The interviews lasted approximately 1 hour. Table 4 illustrates the 
number of interviews conducted per program. 



Table 4 

Number of Interviews by Position 



Content Area 


Project director 
interviews 


Site coordinator 
interviews 


Instructor 

interviews 


Other 

interviews 


Total 


Reading 


11 


16 


30 


6 


63 


Math 


7 


14 


21 


5 


47 


Arts 


9 


16 


31 


15 


71 


Science 


9 


13 


23 


11 


56 


Technology 


8 


15 


26 


5 


54 


Homework 


10 


14 


19 


4 


47 


Total 


54 


88 


150 


46 


338 



The interviews were audio taped to ensure data accuracy, although participants were 
given the option to decline having their interviews recorded, none did. Interview questions 
were focused on topics related to (a) professional background and/or professional 

7 

This also includes project managers, academic and community liaisons, community partnership managers, 
school day principals, tutors, etc. 
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development and experience with afterschool programs; (b) program content knowledge, 
curriculum, and instructional methodology specific to instructional practices in the six 
content areas; (c) internal site structure and organizational characteristics of the afterschool 
programs; (d) external instructional communication and support; and (e) program evaluation 
and areas of perceived program impact. The interview protocol varied slightly depending on 
the interviewee’s position. For example, project directors were not asked questions specific 
to curriculum implementation or instructional strategies, whereas instructors were. The site 
visit team also had the option of conducting additional interviews depending on the specific 
characteristics of a given program or site. 

Once the taped interviews were completed and transcribed, CRESST researchers 
created codes and sub-codes using an inductive approach to analyze the data. A general 
review of the transcriptions informed the development of an initial code set that reflected 
salient concepts and common responses across programs, sites, and respondents. Researchers 
then used the initial code set to qualitatively analyze a small subset of interviews. Upon 
completion of these tasks, researchers condensed initial codes and developed additional 
codes to better reflect the data. Coding reliability was attained through researcher consensus. 
CRESST used the Atlas.ti statistical software package to code the interviews. 

Quantitative Methodologies 

Surveys 

All surveys were distributed to the programs prior to the site visits. The programs 
assisted in the administration of the survey instruments, which were then collected by the 
NPQAL team at the time of the site visits. Surveys were administered to parents, day-school 
teachers, and afterschool program staff including site coordinators, instructors, program 
workers, playground workers, activity coaches, partner organization staff, program experts, 
counselors, and volunteers. Completed surveys from each program were returned to CRESST 
and results were entered into a statistical software program, Statistical Package for the Social 
Sciences (SPSS, 2007). Frequencies and means were calculated according to indicator 
variables (e.g., program structure, linkage to the day school, professional development). 
These results were triangulated with the qualitative data to help provide further evidence of 
“promising practices.” 

There was substantial variance in the response rates across the 53 programs and 
variation in the number of staff members (ranging from 70 to 233), parents (ranging from 
154 to 1000), and day-school teachers (ranging from 62 to 150) who completed surveys. The 
end results and the distribution of response rates for the programs are presented in Table 5. 
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Table 5 



Survey Response Rate 



Content area 


Staff surveys 


Parent surveys 


Teacher surveys 


Math 


74 


377 


NA 


Reading 


233 


1000 


NA 


Arts 


82 


254 


62 


Science 


103 


496 


150 


Technology 


70 


319 


105 


Homework 


83 


154 


107 


Total 


645 


2600 


424 



Descriptive statistics were used to present the frequencies and means of the survey 
responses. Exploratory factor analyses were conducted and constructs were extracted from 
the surveys as well as the APR/PPICS teacher surveys for the purposes of program validation. 
It is also important to note that some content areas may be underrepresented or heavily 
represented due to the varying response rate among the programs. 

INTERNAL PROGRAM STRUCTURE 

This section provides descriptions on the internal program structure of the 53 programs 
in this study, including details about program organizations, program governance and 
leadership, durations and frequencies of instructions, staff education and experiences, 
professional development opportunities, and program resources. The survey data were 
analyzed to capture commonalities and differences among the 53 programs, and to extract 
practices that appeared to align with quality afterschool programming. 

Program Organization 

Figure 3 illustrates survey responses from the staff’s perspectives. Staff members were 
asked to rate their program’s organization on a scale of 1 ( strongly disagree) to 5 ( strongly 
agree). 
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5 



4.5 
4 

3.5 
3 

2.5 
2 

1.5 
1 

0.5 



Students 
understand 
the standards 
for behavior 



The site 
adresses 
students' 
behavioral 
issues 



I regularly 
adjust 
the program 
based on 
student 
learning 



Staff members 
meet with 
each other 
regularly to 
discuss the 
curriculum 



Staff have 
a voice in 
decisions 
about 
curriculum 
and instruction 



Staff take 
an active 
role in 
program 
leadership 



Staff 
receive 
adequate 
support 
from the 
supervisor 



Adequacy 

of 

materials/ 



Figure 3. Program organization. 

In general, the staff across all 53 programs offered high ratings for topics related to 
program organization. They expressed high levels of agreement with the following 
statements (mean level of agreement above 4.0): (a) students understand the standards for 
behavior; (b) programs address students’ behavioral issues promptly; and (c) program has 
adequate materials and resources. Survey results also indicated that staff members felt they 
had a voice in curriculum development, and received adequate support from their supervisors. 
The lowest rating which was still quite high was that the staff members met with each other 
regularly to discuss the curriculum. Interview data further revealed that almost all programs 
conducted bi-weekly or monthly staff meetings, but meetings specifically focused on 
curriculum contents were conducted less often. 

Student Discipline 

Overall, there were minimal disciplinary issues for these programs and the majority 
emphasized that their students understand the program rules for behavior. Staff members 
reported that they agreed that students understood the standards for behavior (mean level of 
agreement = 4.4). Moreover, staff members on average strongly concurred to the statement 
that their site addressed student behavioral issues in a timely and consistent manner (mean 
level of agreement = 4.5). The site observation report supported these findings, and the site 
visit teams reported that the overall classroom climate in these programs was positive, staff- 
student relations were nurturing, staff and student expectations for success were high, and 
disciplinary concerns were minimal. 
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The degree of specificity described by programs regarding their approach to discipline 
varied somewhat. Some programs provided specific examples of behavior problems and how 
they would be addressed, whereas others provided fairly general descriptions of their 
disciplinary policy. Programs also varied in their level of adherence to the regular day-school 
discipline policy. Several programs adopted the same disciplinary system used by the day 
schools they served. Others seemed to believe that afterschool discipline policies should be 
slightly more lenient than the day-school policies, given the students’ energy level and 
attention span after a full day of school. For instance, one afterschool art instructor stated that 
she encountered less behavioral problems than day-school teachers because her students 
responded better to the lenient atmosphere of the afterschool program. 

Program Governance/Leadership 

Afterschool programs are often characterized by the degree to which they adhere to a 
specific decision-making structure. Generally speaking, those with a centralized leadership 
style reserve the majority of the decision-making authority for upper management, whereas 
programs with a decentralized structure are more democratic, allowing and sometimes 
encouraging participation by a wider circle of stakeholders. 

Across all 53 programs in the sample, survey data regarding decision-making processes 
consistently aligned with the interview data. The majority of both site coordinators and 
instructors at all programs indicated in their survey responses that afterschool instructors had 
an active voice in decisions about curriculum and instruction (mean level of agreement = 
4.4), and took active roles in program leadership and decision-making (mean level of 
agreement = 4.1). 

The amount of ownership instructors felt toward decision-making seemed to be related 
to the kind of program culture and expectations conveyed to them by site coordinators and 
project directors. Some site coordinators appeared to be stricter than others in maintaining 
their authority over the decisions made at their sites. But in the majority of cases, site 
coordinators and project directors acknowledged the importance of their instructors to the 
day-to-day operations, especially when they considered the instructors’ familiarity with the 
students and their needs. Interviewees at most programs reported that instructors’ input was 
highly valued and frequently determined the end result, as instructors were considered the 
“experts” in their content area. An instructor of an arts program explained, 

My experience has been that the afterschool teachers propose something that we'd like to 
do with the kids after school, and [the director] then just talks to us about what our plans 
are. We kind of report to her in terms of lesson plans and how we incorporate standards 
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and benchmarks, but a lot of freedom is really given to us. We teach what we’re 
comfortable teaching and what we're passionate about. 

One site coordinator expressed appreciation for the staff by stating, “These are 
professional adults... and they are the best ones to implement the curriculum, because they 
see. They’re with the students, right there with them, and they know what their levels are and 
what their abilities are.” 

In the case of programs where decision-making was characterized as decidedly 
decentralized or “democratic,” administrators consistently described the value of staffs 
content-specific skills and expertise, and as a result, curriculum development was more of a 
group process in which staff members were given a great amount of leeway in designing 
instruction. A project director at a technology program clarified that the latitude his staff had 
was evenly balanced by a strong level of accountability for their curricular choices. 

Obviously, we try to be more democratic.... So one of the things we try to do here, how 
we want to make this a great place to work, is in finding great people, then giving the 
people the power and leeway and the accountability, but also the freedom to do what they 
think works best, and trusting them.... Every quarter they have to come back to us and tell 
us how they're doing. They report back as to what is going on at their... programs. In 
terms of actually decision-making and setting goals and deciding what we're going to do, 
that’s much more of a bottom-up process. 

Overall, data across programs indicated that whether decision-making regarding 
curricula design followed a top-down or a more collaborative model, it was very often guided 
by levels of expertise among staff members. 

Opportunities for Practice 

Miller (2003) states that successful afterschool programs provide activities that enable 
them to gain knowledge and to practice knowledge learned in school, as well as opportunities 
to reflect, make decisions, and solve problems. Similarly, the U.S. Department of Education 
and U.S. Department of Justice (2000) found that students need the opportunity to practice 
and develop their literacy skills through intelligent discussions with adults and peers, 
storytelling, reading and listening, games, and other activities. 

Duration and Frequency of Instruction 

The study examined the duration and frequency of instruction offered at the identified 
programs to ensure that students received substantial instruction time and opportunities to 
practice. Study findings revealed that majority of the programs reported offering three or 
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more activities each day. Most included some kind of homework help or tutoring as part of 
their programs, but other activities offered range from academic (e.g., math, reading, writing, 
science) to enrichment (e.g., arts and crafts, cooking, gardening, health and nutrition, cultural 
activities, computers) and recreation (e.g., sports, dance, drill team, outdoor games). The 
frequency and duration of instruction offered by the programs are provided in Table 6. 



Table 6 

Duration and Frequency of Instruction by Content Area 



Content area 


Average duration of 
daily instruction 


Number of days 
offered per week 


Reading 


51 min. 


3.20 


Math 


66 min. 


2.58 


Science 


77 min. 


2.28 


Art 


77 min. 


3.72 


Technology 


105 min. 


3.55 


Homework 


45 min. 


4.00 



As shown in Table 6, the duration of content specific instruction reported by 
interviewees varied from 45 minutes to a little over an hour per session. All programs offered 
their content- specific instruction at least two times a week. Homework programs reported 
having the least number of minutes (45 minutes) dedicated to homework assistance on a daily 
basis. However, these programs also offered homework help more frequently (4 days a week) 
than the other content areas. Technology programs reported having the most number of 
minutes (105 minutes) dedicated to activities using or teaching technology and offered 
technology instruction an average of nearly four times per week. Technology instruction was 
unique from the other content areas because, although it was often taught as a discreet 
course, it was also consistently integrated into other content areas. For example, the 
afterschool staff reported using technology on a regular basis for academic instruction, as 
well as during enrichment and recreation activities. 

These findings suggest that students were receiving adequate exposure to the 
instructions and were given time to practice their skills. Site observation reports across the 
programs also indicate that students appeared to be mostly engaged and attentive, and 
enjoyed the activities. 
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Staff Education, Training, and Professional Development 



Scott-Little, Hamann, and Jurs (2002) state that successful afterschool programs are 
characterized in part by having well-qualified and well-trained staff. Fashola (2002) notes, 
“Academic subjects taught during the afterschool period require qualified, preferably 
certified, instructors familiar with and who can be held accountable for student outcomes” 
(p. 60). Similarly, the U.S. Department of Education and U.S. Department of Justice (2000) 
believe that it is important to provide professional development to staff members to increase 
their ability to develop and implement developmentally appropriate curriculum, improve 
their skills in supporting and encouraging curiosity and exploration, support their function as 
role models, foster their ability to instill a healthy self-image in students, and to more 
generally attract and retain high-quality staff members. 

Professional development can also give employees ideas for enrichment and hands-on 
activities; greater expertise in academic subject matter; knowledge in assessing student 
progress; and strategies for the different program components of academics, enrichment, and 
recreation. Staff education, professional development, and attitudes are thus important 
program features in afterschool settings. 

Staff Experience and Education 

Through interviews and surveys, site coordinators and instructors at the 53 programs in 
the study sample were asked about their overall experience with afterschool programs and 
their credentials to teach or administrate. Responses gathered from all of the content- specific 

o 

programs were very similar. Of the 150 program staff members for whom these data were 
available, 9 the majority (43%) had experience in afterschool for 3-5 years. In general, survey 
results were compatible with the interview data — although interview data tended to indicate 
slightly higher levels of staff experience. The results for instructor experience are displayed 
in Table 7. 



g 

Number of instructors by content area: Reading (N = 30), Math (N = 21), Arts (N = 31), Science (N = 23), 
Technology (N = 26), and Homework (N = 19). 

9 

Due to unknown reasons, not many of the site coordinators completed a survey; therefore survey data for only 
the instructors was examined for the programs. 
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Table 7 

Staff Experience in Afterschool (N = 150) 



Years of experience in afterschool 


Instructors 


Less than 1 year 


13% 


1-2 years 


15% 


3-5 years 


43% 


6-9 years 


14% 


10+ years 


9% 


Did not report 


6% 



In addition to overall experiences in afterschool programs, program staff members were 
asked about their experiences at the current site. The results indicated that at these programs, 
majority of the staff stayed over 3 years. Math and reading program staff members had an 
average of 3.5 years of experience at the current site. Approximately 75% of the science, 
technology, and arts program staff had between 1-7 years of services at the current site, with 
46% of science, 42% of the arts, and 38% of the technology staff worked over 4 years at their 
current site. Homework programs had 60% of the staff members at the current site for 1-7 
years and 30% from 4-7 years. Because staff stability is important for relationship building, 
especially for at-risk students, staff members can serve as constant figures and mentors in 
their lives. These relationships can provide the basis for students to build trust, character, and 
efficacy, which are all essential elements for good citizenship (Huang et al., 2007; Huang et 
al., 2006). Specific percentages for staff experiences at the current site are available in 
Appendix A. 

Program staff members were also queried about their certifications. Of the 78 staff 
members 10 that responded, 52% reported some type of teaching certification or professional 
development (e.g., credential, Bachelor’s or Master’s degrees). Programs that were district 
affiliated responded that most or all of their staff members were certified teachers, often at 
the day school attended by afterschool students. At these programs, some also held additional 
credentials such as ESL, reading, math, or special education specialists. Community-based 
programs generally hired from the community; there were few certified teachers, staff 
members were typically community members or college students. These findings support the 
results from the staff instructor surveys. 



10 Number of instructors by content area: Reading ( N = 8), Math ( N = 11), Arts ( N = 13), Science ( N = 17), 
Technology (N = 15), and Homework (N = 14). 
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Figure 4 presents the staff survey responses regarding the educational background of 
the program instructors. 




To summarize, the majority of program staff reported having a Bachelor’s degree 
(47%), and 24% had a Master’s degree. In terms of content-specific areas, nearly 45% of the 
math and reading instructors reported having a Bachelor’s degree, and approximately 20% 
had earned a Master’s. Similarly, the majority of arts and science instructors also reported 
having a Bachelor’s degree (34% and 56%, respectively), and 28% of the arts instructors and 
36% of the science instructors reported having earned a Master’s degree. In addition to a 
Bachelor’s or Master’s degree, 10% of arts instructors and 14% of science instructors also 
had obtained a teaching credential. As for technology and homework instructors, the majority 
of technology instructors received an Associate (46%) or a Master’s (31%) degree while 23% 
had earned a Bachelor’s degree. The majority of homework instructors received a Bachelor’s 
(36%) or a Master’s (28%) degree, while 16% of the homework instructors had also received 
a teaching credential. 

Compare to the general afterschool field, the educational level of these afterschool staff 
are very high. This high level of education may have played a role in curriculum design and 
dissemination, which subsequently may have influenced students’ academic outcomes. 
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Professional Development 



The most common types of professional development offered across programs were 
general techniques in working with students, such as classroom management, discipline 
issues, social issues (e.g., cooperation or bullying), and general teaching strategies. 

A few programs had professional development that was tailored to the specific needs of 
their student population. For instance, one program provided professional development on 
gang awareness/prevention, and a couple others focused on preparing staff to serve students 
with special behavioral needs such as ADHD, or academic needs related to second-language 
issues. Two science programs offered professional development on child abuse and poverty, 
some math and reading programs offered trainings on helping students in math and reading, 
and others had staff development for integrating technology and computers. Some programs 
also provided instructors with professional development on how to motivate and connect 
with students. 

Although all interviewees reported having some sort of professional development 
available through their afterschool program, most program staff members received the bulk 
of any formal or semi-formal content- specific professional development through the day 
school or the district. More specifically, the district-affiliated programs generally encouraged 
staff to attend district professional development together with the regular day-school 
teachers. Among the six content areas, the technology program was the one that offered 
professional development to their staff consistently. While one technology program director 
reported offering extensive, rigorous, and regular technology professional development 
exclusively to afterschool staff (i.e., not affiliated with the day school or the district), staff at 
the majority of technology programs also supplemented staff development with self-sought 
or peer-to-peer technology professional development. 

When opportunities were available and staff participated in the professional 
development, they generally found it useful. Instructors particularly appreciated professional 
development that was directly applicable in the classroom, relating to teaching strategies, 
innovative approaches and classroom management. As one program staff explained, “That’s 
the best part — that we can actually incorporate it. You don’t just learn it to say you know it, 
but to actually do it with students.” 

Staff responses also indicated that professional development aided in practical 
knowledge that led to promotions. Some afterschool staff characterized professional 
development as a means of realizing long-term professional goals, as illustrated by one 
program staff, 
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It has definitely been one of the most beneficial things that we have. I believe that it has 
helped our staff members grow as far as within the program and for their own 
development. Most of the staff members that I've worked with or that I've known usually 
want to become teachers and so this is a great stepping-stone for them. 

Staff Participation in Professional Development 

Although opportunities of professional development were offered to the program staff, 
they did not necessarily participate in it. Although 57% of the staff reported that their 
programs offered professional development two to four times a year (Table 8), only 26% 
participated in professional development two to four times a year. Sixty percent of the staff 
reported that they never participated in the professional development offered through their 
programs, and 14% reported that they participated once (Table 9). When we desegregated the 
data by content, the highest participation rate was from the technology instructors; 50% 
indicated that they had participated in professional development two or more times a year 
whereas 42% responded that they had never participated in professional development for 
technology instruction. 

Table 8 

Professional Development (Offered by Site) 



Frequency 


Percent 


Never 


18 


Once 


12 


2-3 times 


22 


4 times or more 


35 


Don’t Know 


13 



Table 9 

Professional Development (Participated by Staff) 


Frequency 


Percent 


Never 


60 


Once 


14 


2-3 times 


17 


4 times or more 


9 


Don’t Know 


0 
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While examining the data, a pattern emerged. Notably, project directors and site 
coordinators seemed to report higher frequencies of participation in professional 
developments than the program staff. There are two likely explanations for this incongruity: 
(a) Many project directors were not clear in differentiating the professional development that 
was offered to site coordinators and other members of upper management, as opposed to 
program staff at sites, (b) It also seemed that program staff had a more rigid definition of 
what counted as formal professional development, whereas many site coordinators counted 
weekly staff meetings as professional development. At the same time, it also appeared that 
site coordinators and staff in upper management generally received the majority of the off- 
site professional developments. The following Figure 5 illustrated an example from Year 1 
reading and math. 




Figure 5. Types of professional development offered to math and reading grantees. 



In Figure 5, similar to the interview findings, it is observed that a higher percentage of 
site coordinators reported receiving professional development in most categories (other than 
working with English language learners), and instructors reported receiving the lowest 
percentage of professional development in program and self-evaluation. Similar trends were 
observed for science. Due to the low return rate for site coordinator surveys, site 
coordinators’ responses were not included in the analyses for arts, technology, and 
homework. 



29 



Variations in Professional Development According to Content Areas 



The types of professional development offered at the afterschool programs also varied 
according to their content focus. Math and reading program instructors cited similar types of 
professional development offered at their afterschool programs (see Figure 5). Most 
professional development opportunities offered were on helping students with reading and 
math, applying state standards to the curriculum, and connecting with the day school. Fewer 
opportunities were offered on working with English learners, self-evaluation, and 
assessment. 

Professional development specific to the arts ranged from an artists’ retreat at which 
instructors could share their work and discuss best practices; dealing with conflict resolution 
and how it relates to the arts; to staff development on specific topics such as music, dance or 
paper-making. More specifically, the majority of art instructors (59%) indicated that they 
received professional development on helping students with art projects. It should be noted 
here that five out of the nine arts programs had local partnerships with arts studios and 
experts that served as instructors for these programs. 

Over half of the staff interviewed at the nine science programs (69%, N = 58) reported 
that professional development was offered regularly at their sites. The types of professional 
development offered across programs focused mainly on coverage of science curriculum and 
instruction. Specific professional development sessions ranged from week-long science 
workshops to creative ways of exposing children to science through play and exploring; new 
ways of teaching different areas of science; and staff development on specific topics such as 
pollution and the environment, chemistry, and earthquakes. 

The majority of the technology programs reported that implementing or maintaining 
regular professional development sessions was a goal of the program. All interviewees 
reported having some sort of technology-related professional development available through 
their afterschool program. However, it should be noted that at 6 of the 10 programs, staff 
received the bulk of any formal or semi-formal technology professional development through 
the day school or the district. Only one program director reported offering extensive, 
rigorous, and regular technology professional development exclusively to afterschool staff 
(i.e., not affiliated with the day school or the district). A majority of the staff also 
supplemented formal staff development with self-sought or peer-to-peer technology 
professional development. That is, a good number of interviewees sought professional 
development on their own (e.g., through conferences or online), and many also received 
technology instructions from more knowledgeable colleagues in the afterschool. 
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Meanwhile, about 50% of the homework program staff reported that no professional 
development was offered to them. Of those who received professional development, 28% of 
the program staff indicated that they received professional development on evaluating the 
program and self-evaluation. Twenty-two percent of the staff responded that they received 
professional development on connecting afterschool learning with day-school curriculum, 
and 20% of the staff reported receiving professional development in techniques for helping 
with homework. 

Adequacy of Space and Resources 

Program staff were asked about their needs in terms of resources (i.e., supplies, staff, 
space, etc.) in the afterschool program. Consistently across programs, a need for additional 
space was most commonly mentioned by both site coordinators and program staff. Many 
programs seemed to rely on access to common space, such as an auditorium or a classroom 
shared with day-school teachers, which often caused logistical problems and sometimes 
prevented planned activities from taking place. Furthermore, some programs expressed 
difficulty with not having consistent access to classrooms. A site coordinator illuminated the 
problem, “I would say physical space would be definitely a big thing with our 
program.... That is probably one of the hardest things to work with just because every 6 
weeks we are readjusting the classroom to new classroom seating charts, new areas in which 
the students can and cannot go.” Additionally, several interviewees commented on the need 
for a separate office and storage space. 

According to most programs, another scarce resource was access to technology and 
particularly computers. Although all the technology programs in this study indicated they 
have sufficient computers and tools to work with (it should be noted here that many 
technology programs were funded by, had a partnership with, or received support from 
technology companies such as Dell, Apple computer, etc.), other programs were frustrated by 
old equipment and a lack of current software. Although survey results indicated most of the 
programs have sufficient materials and supplies to work with, a few interviewees also 
requested more access to textbooks, literature, and general supplies. Site coordinators at a 
few programs also indicated that they would utilize additional funds they raised through fund 
raising and other activities not only to purchase supplies, but also to hire staff with expertise 
in the field of education or social work due to program popularity and growth. 

Summary of Internal Program Structure 

The internal program structure of these 53 programs followed a similar pattern. The 
staff in these programs came with 3-5 years of afterschool experiences, and had lower 
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turnover rates with an average of 3.5 years working at their current sites. Most of them also 
came with either a Bachelor’s or Master’s degree, especially for the science, arts, and 
technology programs. These high levels of education may be attributed to the fact that arts, 
science, and technology are much more content specific, and these instructors may have 
pursued this as their area of professional interest. 

In general, all programs offered professional development at least once or twice a year 
but staff participation was low. For example, 35% of the afterschool staff reported their 
program offered professional development over four times a year, but only 9% of the staff 
participated in these professional development four or more times a year. Over 58% of the 
staff indicated they never participated in professional development offered by the program; 
and 32% reported they rarely participated. Technology programs had the highest 
participation rate, with majority of the technology programs emphasizing the importance of 
following the recent developments in technology. 

All programs provided initial professional development on student discipline and 
classroom management. School district related programs tend to provide greater 
opportunities and varieties for professional development, having the afterschool staff 
participate in the professional development together with the day-school teachers, and the 
participation rate is generally higher. When the program staff participated in the professional 
development, they generally reported that it was useful, especially when the professional 
development sessions were directly applicable to the afterschool classrooms. 

As for program organization, the programs were generally well managed, usually under 
strong leaders who were able to articulate a clear program mission and vision statement and 
goals, and to empower and motivate the staff to achieve the program objectives. The students 
appeared to understand the codes of conduct for behavior; when necessary the programs 
address students’ behavioral issues promptly. These programs also had adequate materials 
and resources. 

In these programs, staff felt they had a voice in curriculum development, and they 
received adequate support from their supervisors. In a majority of the cases, site coordinators 
and project directors acknowledged the importance of their program staff to the day-to-day 
operations, especially when they considered the program staff’s familiarity with the students 
and their needs. This trusting relationship empowered the staff to set high expectations to 
their students and developed efficacy in their instructional activities. The team culture that 
existed in most of these programs indicated that positive relationships and communication 
among the afterschool staff appeared to enhance their ability to expand their roles; and the 
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frequent interaction among program staff also appeared to have a positive impact in fostering 
problem solving, thus decreasing the level of assistance sought from the site coordinator. 

As for scheduling, the duration of content specific instruction reported by interviewees 
varied from 45 minutes to an hour and 45 minutes per session. All programs offered their 
content- specific instruction at least two times a week. Technology programs reported having 
the most number of minutes (105 minutes) dedicated to activities using or teaching 
technology and offered technology instruction an average of nearly four times per week. 
Technology instruction was unique from the other content areas because, although it was 
often taught as a discreet course, it was also consistently integrated into other content areas. 
For example, afterschool staff reported using technology on a regular basis for instruction in 
academic content areas (e.g., math, reading, writing, science), as well as enrichment (e.g., 
arts and crafts, cooking, computers) and recreation (e.g., sports, dance, outdoor games). 
These findings suggested that students were receiving adequate exposure to the instructions 
and were given time to practice their skills. Site observation reports also indicated that across 
the programs, students appeared to be mostly engaged, attentive, and enjoying the activities. 

PROGRAM PROCESS 

This section provides an overview of the process for program implementation at the 53 
programs. Descriptive findings on parent participation, connecting with the community, 
staff-student interaction, and student engagement in terms of opportunities for practice, 
social development, and motivational support are included in the following sections. 

Parent Involvement 

According to the literature, parent involvement is one essential indicator in external 
connection and support for exemplary practices in afterschool programs. Site coordinators in 
the study sample were inquired in interviews and surveys about their perceptions of parental 
involvement in the afterschool programs. Distinct themes emerged across both survey and 
interview data for all 53 programs, with very few exceptions or variations. 

Most programs shared similar methods of disseminating information to parents, as well 
as means of encouraging their involvement in the afterschool programs. However, despite 
great efforts in these areas, almost all programs showed evidence of fairly low parent 
participation or volunteerism, as well as low percentages of formal parent-teacher meetings, 
given that most of the programs also did not have a formal structure in place for scheduling 
regular meetings with parents. More details are provided in the following sections. 
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Sharing Information and Promoting Participation 

Site coordinators and program staff across all 53 programs consistently offered a broad 
range of means by which they shared program and student progress information with parents. 
Some of the most frequently cited means of communication were: monthly/bimonthly 
newsletters; pre-academic year orientation; parents’ nights; community outreach activities 
such as potlucks and student performances; afterschool nights throughout the academic year; 
letters, notes and phone calls home as necessary regarding student progress, attendance, or 
behavioral issues; and scheduled meetings with site coordinators or program staff as 
necessary. 

Figures 6 shows survey results indicating parents’ perceptions of program efforts to 
share information and promote participation. Overall, responses indicated that parents felt the 
programs made an effort to keep them abreast of program rules and practices, and to 
encourage their participation. About 20% of parents indicated that the program never invited 
them to volunteer in the afterschool program, but nearly 40% indicated that they were invited 
to volunteer at least once a month. About 35% of the parents surveyed stated that their 
children’s afterschool programs provided program information in their home language at 
least once a month. 




□ Never 

□ Once or twice a year 

□ Once every few months 

□ At least once a month 



Someone at the program 
explaninsto me the program's 
rules 



The program invites me to 
volunteer in the program 



The program sends me 
handouts/i nf ormati on i n my 
home language 



Figure 6. Parent awareness of the afterschool program. 
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Similarly, over half of the programs mentioned in interviews that all parents were 
encouraged to volunteer at their sites in some capacity (e.g., in the classroom, on field trips, 
chaperoning events), and a few sites also mentioned offering weekend parent workshops 
(e.g., crafts, cooking, ESL, or GED courses). Despite these efforts, estimated percentages of 
actual parent involvement were extremely low or nonexistent across the programs. The great 
majority of interviewees acknowledged that although they encouraged parent participation in 
their program, volunteerism was extremely low. The key reason appeared to be that they 
predominantly served families where both parents worked and thus were unavailable to 
afterschool activities on a regular basis, if at all. As one program staff at a homework 
program explained, 

A lot of them [parents] are interested to know what’s going on... but as far as coming in 
and checking it out, it is a very small percentage. They do get the newsletters and the 
calendars so they do know what we’re up to. Most parents [however] are working at that 
time; so it makes it hard for them. 

Parent survey results, as shown in Figure 7, confirmed that the frequency of parent 
participation was low across all programs. 




Do you visit Do you help out Do you receive a Does the Do you attend 
your child's in your child's newsletter from program offer a parent 
afterschool afterschool the afterschool parents' night workshops or 
program? program? program? for you? classes offered 

by the 
program? 



□ Never 

■ Onceortwiceayear 

□ Onceeveryfew months 

□ Atleastonceamonth 

■ Morethanonceamonth 



Figure 7. Parent involvement at the 53 programs. 



35 



Over half of the parents reported receiving frequent, once a month or more, newsletters 
from their children’s afterschool programs. However, just over half of parents stated that they 
never help out at their children’s afterschool programs; and the same percentage reported a 
failure to attend parent workshops or classes offered by the program. 

Parent Meetings 

Survey results also suggested that afterschool staff did not meet with parents on a 
regular basis. As shown in Figure 8, afterschool staff across the programs reported infrequent 
meetings with parents. Almost 50% of the staff said that they never or rarely had formal 
meetings with the parents, only 19% of staff meets with parents on a regular basis. 




| □ Instructors 

Figure 8. Frequencies of parent meetings at the 53 programs. 

Interview responses regarding formal parent meetings with afterschool staff indicated a 
similarly low occurrence. Although some programs had a formal parent meeting scheduled at 
the onset of the school year (typically a group orientation rather than a one-on-one meeting), 
a significant number of afterschool staff stated that they rarely, if ever, met formally with 
parents. However, almost all staff in the 53 sites mentioned that they frequently took the 
opportunity to speak and discuss student issues with the parents when they picked up their 
children in the evenings. 
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Parent Perception of the Program 




I feel welcomed Someone is I feel comfortable Staff helps me I am notified Staff deals with Staff cares about There are staff 
tovisitmy available to talk asking about understand immediately if my mychild's and respects my memberswho 

child (ren) tome what my child is school child isn't paying behavior child speakmyhome 

learning documents attention to problems quickly language 

school work and fairly 



Figure 9. Parent perception of the program. 

Although formal meetings and involvement with parents were rare, the parents’ 
perceptions of the program were very high. Ratings were based on a 10-point Likert scale 
1 {never) to 10 {always). Parents indicated that they felt welcome to visit their children’s 
afterschool programs at any time, and that there were staff members available to speak to 
them in their home language. They felt comfortable talking about program materials, and felt 
that the staff cared about and respected their children. They also reported that afterschool 
staff dealt with their children’s behavioral problems promptly, and that they were notified in 
a timely manner if their children were not paying attention. 

Interestingly, a pattern of progressively increasing parent satisfaction emerged over the 
3 years. It should be noted that only quantitative data that were uniform across all six content 
areas were examined. Averages for each year were calculated and then compared across the 3 
years. As illustrated in Figure 10, it appeared that parents felt progressively more satisfied 
with their children’s afterschool program over the 3 years. There was a small increase from 
Year 1 to Year 2, and a higher increase in Year 3 on all four ratings. In reading this table, one 
should bear in mind that the content areas under study were different in these 3 years. This 
heightened satisfaction could be a result of: content focus, time, or improvement in 
afterschool functioning, or any combination of such. 
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10 




OYearl 

■Year2 

□Year3 



The kind of Performance of What child learns The resources 
activities offered the after-school in after-school that the program 
staff provides for 

parents 



Figure 10. Parent’s satisfaction w/ their child’s afterschool program. 



Connecting with the Community 

Many of the 53 programs mentioned ways in which they connected with the 
community. The activities most frequently mentioned were through service learning, such as 
making get well cards for or reading to patients in the hospitals, taking trips to nursing homes 
to sing to the senior citizens, recycling or community beautification efforts, and cultivating 
community gardens. One afterschool program worked with a local charitable organization 
not only to restore a public garden, but also hike, take horticulture classes, go rowing, and 
swimming in the lake. An instructor at another site described a local shopping campaign that 
entailed giving kids a budget and bringing local storeowners into the school to set up a mini 
version of the economy; and another instructor at the same site had a pet store owner come to 
the school and build an entire mock ecosystem within the building to teach kids about 
biology and conservation. 

Another way in which the community was involved in the afterschool programs was 
through resources. Most staff members reported benefiting from various types of support and 
involvement from the local community. A broad range of community connections that served 
to enhance learning was reported. Benefits from these partnerships were especially valuable 
for arts, science and technology programs where supplies can be costly. For instance some art 
programs benefited from donated materials and supplies, funding, arts-related outings, and 
artists-in-residence programming. 
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Many programs also reported getting volunteers as a result of their affiliations with 
local universities and high schools. Other programs get volunteers from community 
organizations such as Boys/Girls scouts, churches, and boys and girls clubs. For instance, in 
homework help programs many of the staff mentioned that volunteers from local universities 
and colleges were able to get involved in tutoring students. Furthermore, afterschool program 
staff also frequently invited science experts from the community to visit their program to 
offer students a more tangible experience of practical, real-world applications for science. As 
one project director explained, 

What makes it unique at [our program] is we have so much community involvement in 
teaching science... We’ve really tried to get experts in the field to come in... I don’t think 
that there is any program that has more community involvement in teaching students 
science than ours. 

The importance of bringing community members into the program was echoed by a site 
coordinator from an art afterschool program: 

That’s the beautiful thing about coordinating the afterschool program. My job is really to 
find these community-based organizations and individuals that that’s their passion. 

That’s what they like to do, whether it’s art or drawing, cartooning, dancing, singing, 
whatever it is. You're actually bringing in people in the community that really enjoy 
what they do and want to teach that to kids. 

By bringing community members and resources into afterschool programs, students are 
provided with opportunities to explore interests, identify role models, and become more 
immersed in their community. From the varied ways in which afterschool programs enrich 
their students’ learning experience, it appears that afterschool programs have a strong 
commitment to providing enrichment activities to strengthen students’ self-concept, 
character, creativity, and community immersion. 

Relationship Building with the Students 

Afterschool literatures continue to emphasize the importance of the contribution of high 
staff expectations and positive attitudes toward students to the quality of the program 
environment and student achievement. Hall, Yohalem, Tolman, and Wilson (2003) state, 
“The staff of effective afterschool programs intentionally create a culture of high 
expectations that affirms the potential of each participant and communicates clear 
expectations and standards concerning participation and behavior. ..high expectations 
combined with opportunities to meet those expectations leads to increased motivation and 
engagement.” This study examined the quality of staff-student interactions in the 53 
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programs. The site observation field notes and reports indicated that positive relationships 
and interactions between the staff and the students were observed in all of the 53 programs, 
particularly in terms of expectations for student performance, disciplinary issues, and 
democracy. 

Expectations for Students 

Program staff were inquired about their expectations for students’ performance. 
Although none of the programs responded that their expectations were low, they did differ in 
the ways they framed their expectations. For example, about a third of the programs 
responded that they wanted the students to be performing at grade level or better; and nearly 
one-fourth of programs mentioned wanting students to do their best in academic 
performances. Science and arts programs distinctly expected their students to gain exposure 
and engage in science and arts experiences, whereas technology programs expected their 
students to use technology as a means for continuous learning in different life experiences. 

Democracy in the Classroom 

Although most programs were not designed to gather student feedback or give students 
many choices, especially the reading, math, and homework help programs; staff from these 
programs enhanced students’ sense of democracy by taking their interests into account when 
making decisions on program activities. For example, one program allowed students to offer 
input on where they would like to conduct their service learning projects; others considered 
students’ activities choices, at one site, students were given the option between reading and 
doing homework. 

Perhaps due to the context of the different subject focus: arts, science, and technology 
programs placed a heavier emphasis on students’ autonomy. As an example, an arts program 
staff stated that student inputs are of great value; hence, student interests had a vast impact on 
art curriculum content. These kinds of autonomy help students in taking ownership in their 
learning process and help to keep them engaged. 

Motivational Support: Engaging the Students 

In addition to positive relationships, motivational support is important in supporting 
students’ academic achievement (Bempechat, Graham, & Jiminez, 1999; Ryan & Grolnick, 
1986). Rossi and Montgomery (1994) states that increases in student academic engagement 
are associated with quality resources, as well as the pressure or incentives for students to 
invest in academic achievement. Miller (2003) states that students may be motivated by 
personal interest in a topic, the desire to please a teacher, peers, parents, and other adults, the 
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knowledge that success has long-term rewards, and a desire to increase a sense of their own 
capabilities. Analyses revealed that a variety of motivational strategies were implemented at 
the 53 programs in the study sample. Researchers noted that a variety of motivational 
strategies were implemented at the 53 programs in the study sample. 

Making Learning Fun 

The majority of programs employed unique and innovative strategies to engage 
students in the afterschool setting, placing a particular emphasis on making learning fun. The 
primary reason given for this approach was the need to maintain interest after a long day at 
school. As one instructor aptly stated “Afterschool should simultaneously be fun and 
informative.” To accomplish this, a number of strategies were used including cross-content 
integration, diversity of activities, real world examples, dialogic and cooperative learning, 
culturally significant programming, special consideration for the students’ activity 
preferences and the incorporation of enrichment and recreational activities. Intentional or 
“disguised” learning was also popular with sports, games, discussions, and journal writing; 
and were used to engage learners while pushing learning into the background. A site 
coordinator explained the approach: 

I think that because a lot of our program is disguised learning, a lot of times the kids 
don’t even realize that they are doing math or that they are doing language arts or 
reading. So, in essence, we have them already developing a lot of social skills. During 
our “Why Be Healthy?” time, for example, when they are competing against other 
schools, they are learning team sports and character traits that are appropriate for youth. 

Breaking up the Day 

Diversity in activities was one key way the various programs maintained interest 
among their students. The daily schedule generally comprised a combination of homework 
help and tutoring, academics, and some enrichment and/or recreational activity. Most 
programs had short periods of 30 minutes to an hour per activity, and a few used learning or 
activity centers that allowed children to move from one activity to the next at their own pace. 

Real World Examples 

Programs also tried to engage students by offering real world examples that connected 
the curriculum to their lives outside of school. This was particularly salient among science 
and technology programs; across these programs, hands-on activities with real-world 
connections were the most popular instructional strategies. Over half of the technology 
programs reported engaging students in authentic, real-world, relevant activities; these 
activities frequently led to the development of marketable and applied skills. For example, 
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some technology sites trained students to use software programs to produce music or graphic 
design. Additionally, many science instructors reported inviting science experts to discuss 
real-world applications of science. The math programs also created activities that related to 
everyday life experiences like banking, shopping, and budgeting. As one site coordinator 
stated, “We use things like BMX bikes or other things that interest them.” Overall, most of 
the staff expressed a desire to further engage students by using relevant and real world 
examples. They also accomplished this goal by referring to local places, familiar activities, 
and known individuals. 

Current Events 

Programs attempted to make content more relevant by tying it to current events. A few 
sites indicated the use of newspapers and magazines in class, others mentioned studying 
popular culture like hip-hop and movies, some sites studied popular gadgets, and several sites 
used television as the starting point for an educational activity. Overall, popular culture 
references were readily utilized; on the other hand, national and world news events were 
rarely mentioned. A math instructor highlighted her use of current events: 

When we have reading I like to use, especially on Thursdays when we do current events, 
the daily newspaper from town. 1 usually pick the story, but a lot of times I let them pick 
an interesting one. The last one we did was a big article about the presidential race. So 
we tied that into math. 

Providing Opportunities for Social Development 

Social development was emphasized in varying degrees at all programs, and positive 
socialization was generally observed among the students. When asked about program goals, 
about half of the staff members mentioned social development, and overall, student-to- 
student interactions at the 53 programs appeared to be positive, nurturing, and respectful. A 
site coordinator highlighted the way that all activities are related to social development: I 
don’t think that there’s anything we do that in some way doesn’t impact either their image of 
themselves or how they interact with the world. 

Many strategies were used to enhance social development; the most commonly 
reported strategies included the use of group activities, multi-age groupings, rewarding pro- 
social behaviors, and encouraging peer collaboration. One of the strategies employed to 
promote social development are through the enrichment activities. Some examples of 
enrichments offered included: arts activities, field trips, sports, and character building 
workshops. 
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Art Activities 



Almost all afterschool programs offered arts activities for their students as an 
enrichment including poetry, dance, drama, choir, and drawing. One program instituted an 
innovative program, “Fun with Junk,” where kids created art projects out of recyclables. 
Other sites put on drama, dancing, or singing productions for fellow students, teachers, and 
parents, thus providing opportunities for students to work with and collaborate with each 
other. 

Character-Building Programs 

Many of the 53 programs mentioned providing additional program curriculum that was 
geared toward supporting students’ social development, increasing their self-esteem, and 
creating positive self-images. The site coordinator at a reading afterschool program described 
two programs that they offered for their students: “The girls get Smart Girls, which is also 
character building, but it deals with those life changes during puberty, taking care of your 
body in terms of hygiene, and what does it mean to be a woman in society. Same thing for 
men, for the boys it’s Passport to Manhood.” Another site coordinator for a math afterschool 
program also mentioned a program called Character Development, which focused on 
teaching students values such as “honesty, respect, responsibility, and caring.” 

Field Trips 

Field trips were yet another method used to engage learners and provide real world 
links to the afterschool program subjects. Over half of the programs mentioned taking 
students on field trips. Parents were encouraged to volunteer for field trips; hence the benefits 
were threefold: enhancing education by providing concrete experiences; providing social 
development opportunities as students engaged in planning and discussions of the events; and 
furthering parent involvement. Math and reading programs generally visited museums, zoos, 
libraries and fire stations. Moreover, several programs reported taking students to local 
festivities such as pep-rallies. Art programs, generally, opted for museums and art 
exhibitions, for instance as one art program instructor explained: 

Our program offers a field trip, for example to the Getty [museum], and most of my kids 
will go on that trip. And that’s a really nice time in the sense that we're introducing the 
arts to them. We visualize, and then they have these field trips that correlate it together 
and reinforce what we're doing in class. 

Furthermore, some art programs arranged for a more interactive field trip experience, 
for instance, according to a site coordinator at an art program, students were bused to “the 
Tucson Symphony Orchestra had sort of collaborated and the kids would do field trips into 
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their facility and would actually pair up with a musician.” Science and technology programs 
favored field trips that supplemented their curriculum such as science museums and 
technology centers. For instance, students at one science program were taken on a field trip 
to Lockheed Martin: a leading multinational aerospace manufacturer and advanced 
technology company. 

Summary 

In summary, most parent-staff communication was informal and occurred during pick- 
up time, programs also made use of written forms of communication such as newsletters to 
share information with parents. Parent involvement across the programs was generally low. 
However, they felt that the program did a satisfactory job in keeping them up-to-date with 
program rules and practices; their children were well taken care of, and any disciplinary 
issues were dealt with efficiently and fairly. 

With some exceptions, community involvement was not particularly strong at these 
programs. For math, reading, and homework, community involvement generally consisted of 
service learning opportunities, volunteers recruited from the community, and students 
participated in community events such as paper recycle, beach cleaning, etc. Arts, science, 
and technology programs appeared to have more in-depth collaborations with their 
community through sharing resources, both in materials and in expertise (e.g., arts residence 
programs assigned artists to the afterschool program for a period of time to work with the 
students, Dell computer brought in materials, lesson plans, and technology assistance to the 
afterschool program, and local museums sent scientists to deliver a project with the students, 
etc). Some of them also collaborate with local colleges and universities. 

Staff-student relationships were characterized by warmth and mutual respect 
throughout the 53 programs. Almost all programs wanted the students to improve both 
academically and socially. In building relationships with students, staff in these programs 
expressed high expectations to their students; provided democracy in the classroom so that 
students have a voice and feel empowered; rules were made fairly and observed consistently; 
some programs also provided character building programs; and all programs provide 
motivational support and desired to engage their students through activities. Consequently, 
minimal disciplinary issues were observed or reported. 

Social development was also an important goal to most sites. Staff employed 
techniques such as peer collaboration, different grouping arrangements, and cooperative 
learning opportunities to realize this goal. According to field notes, most student-to-student 
relationships observed at sites were positive. To facilitate learning, student motivation and 
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engagement, programs utilized five major techniques. First, they tried to make learning fun; 
the staff accomplished this goal by including recreational activities and disguised learning. 
Second, the staff broke up the day, and as a result students enjoyed a diverse set of activities. 
Third, the staff used real world and relevant examples to teach the curriculum. Fourth, many 
instructors discussed current events, for example some instructors referred to popular culture 
and events. Finally, about half of programs made use of field trips, student performances, and 
exhibitions to advance learning. 

GOAL SETTING AND CURRICULAR PRACTICES 

This section presents the findings focused on goal setting and curricular practices. Four 
thematic areas will be discussed including setting of program goals, standards-based 
curriculum, links to day school, research-based curriculum and instructional practices within 
specific content areas. 

Goal-Oriented Programs 

Through a meta-analysis of the literature, Bodilly and Beckett (2005) found that the 
setting of clear goals and desired outcomes is essential for program success. More 
specifically, the structure and design of afterschool programs should be guided by the 
purposes of the program. This section will discuss how the afterschool programs aligned 
activities to meet program goals. 

Aligning Strategic Plans to Achieve Goals 

In terms of aligning program activities to goals, all 53 programs appeared to be able to 
structure their curricular design and program practices to facilitate meeting their program 
goals. For example, three of the math programs and four of the reading programs specifically 
targeted students who were struggling academically. The site coordinator at a math program 
described their program goals as focused on developing students’ academic skills within the 
math content area: 

Our primary goals are to bring the student to grade level. That’s my primary goal and 

covering the basics of addition, subtraction, multiplication and division. 

Programs with achievement goals like the earlier mentioned are often district-affiliated, 
they generally ran a more structured program, stressing the improvement of academic 
performance, hiring more certified teachers, and maintaining a more consistent linkage with 
the day school (in terms of curriculum coordination, communication, and adherence to state 
standards). They used motivational strategies to maintain a high degree of student 
engagement, motivation, and parent satisfaction. 
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At the same time, a few reading programs exhibited a strong commitment to their 
community. One collaborated with the YMCA and had strong ties with their community; 
another stressed building citizenship through apprenticeship. To achieve their program goals 
of linking with the community, these programs recruited staff members from the community 
and had activities that connected students with their neighborhoods and communities through 
service learning, field trips, and other activities beyond the classroom instructions. 

Slightly different from math and reading, the majority of the arts programs had a 
different focus. Their common goal was to provide students, who otherwise would have little 
to no exposure to the arts, with art experiences. Program staff, both instructors and 
administrators alike, frequently noted that although the afterschool curricula attended to the 
need for academic achievement and progress, arts instruction could also serve as a means to 
encourage creativity and discovery through expression — something most of their students 
had very little experience with due to recent school cut backs and financial short fall. As one 
site coordinator explained 

These are kids that have never been exposed to anything — I mean really nothing... [our 
goal is to] open the kids’ minds to new things, and to show that they can do it.... We 
wanted a well-balanced, well-rounded program with a lot of different things to offer to 
the children, to open their minds. 

A number of interviewees further suggested that by using an integrative approach to arts 
instruction in afterschool programs they could potentially impact students who were 
struggling academically and personally. 

Similar to arts, all science staff responded to questions about curricular goals by 
suggesting a primary interest in simply offering positive science experiences to students. As 
one executive director remarked, “[Our] primary goal... is to awaken in students a curiosity 
about science and the world around them.” Beyond this basic theme, a smaller percentage 
(just over 20%) of interviewees across science programs characterized their curricula as 
mostly focused on academic improvement and achievement. Most of these respondents 
reported that they considered their programs to be an opportunity to extend science projects 
and labs from the day school to afterschool, as was the case of one site coordinator who 
remarked, “We’re growing on what they’re teaching [in the day school], trying to enrich 
those areas.” A few went even further to explain that their principle goal was to improve 
standardized test scores in science by focusing on extending day-school instruction into 
afterschool. One academic coordinator described designing afterschool science curricula with 
a strong academic focus in order to “bring students up to par for standardized testing.” 
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Technology programs appeared to have their main focus on developing students’ 
applied knowledge. Nearly all of the technology staff interviewed reflected the goal of 
teaching students the mechanics of a broad range of technology skills, in the interest of 
encouraging the students to use those skills to enhance learning in other content areas; and 
teaching the students a technical skill that can be relevant to their real lives, both now and in 
the future. 

Only one technology program mentioned academic achievement as their primary focus. 
In this program, integrating technology into other content areas “to improve reading and 
math scores” was the primary goal, whereas staff across the other nine programs consistently 
explained that academics were just a piece of the puzzle, and development of skills in 
technology could serve their students on multiple levels. One project director explained that, 
by showing students that learning through technology can be interesting and relevant to their 
lives, the program could positively impact students’ futures far beyond elementary and high 
school experiences. 

Everything that we do in afterschool, we try to make it somehow connected to the real 
world.... That’s one of our many goals is to make it relevant, that everything we do be 
relevant... to perhaps make a connection for them for a reason for staying in school, a 
reason to go to college, a reason to be on a particular career path; and if technology is 
what fascinates them then that’s what we will use. 

Interview data across the seven homework programs suggested that the primary goals 
in afterschool homework help were to complete homework and increase academic 
achievement. Many staff members reported that assisting the students to understand their 
homework as well as increasing motivation to complete homework was important to achieve 
their goals. 

We recognized that the other role we have to play is to get children engaged in the 
learning process. So it’s not just about completing the homework, but it’s about finding 
ways outside of the school to get them interested, excited, feeling confident, and to build 
their self-esteem so that even if they only do a third of the homework that they feel good 
about it, and they want to come back the next day and try a little harder. That’s giving 
them some sense of motivation. 

In aligning activities to achieve their program goals, nearly all interviewees described 
the merits of developing a curriculum that combined academic skill development with 
opportunities to explore, and encouraging students’ social development. One program 
director summed up well in saying, 
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Our goal is to help each child to make sure that it’s an afterschool program that’s fun, but 
at the same time it’s enriching, so they grow and learn every day, so they can take home 
some more knowledge. . . 

Standards-based Curriculum 

Through interviews with site coordinators and program staff, questions were asked 
related to their familiarity with their states’ content standards, the ways in which the 
curriculum is linked to the standards, and how coverage occurred. Additionally, program 
staff were asked to explain which specific standards were covered in a lesson observed 
during the site visits. Overall, interviews revealed that most programs appear to be 
incorporating state standards into their curriculum. This occurred across programs to varying 
degrees with some knowledge about, and purposeful in, delivering a standards-based 
curriculum, whereas others appeared to make it a secondary, or less intentional goal. This 
section will discuss staff familiarity with standards, and the ways that standards are linked to 
instruction. 

Familiarity with the standards. Most program staff reported that they were familiar 
with the state standards, although levels of familiarity seemed to vary. Some program staff 
and site coordinators stated that they had a strong knowledge of the standards, often because 
they were also day-school teachers, or because their program built on the day-school’s 
curricular structure. In about half of the programs, especially those that are community- 
based, one or more of the interviewees responded that they were not very familiar with the 
state standards. In most of these cases, interviewees further explained that they had some 
familiarity, or knew where they could find the standards. Also, in a few cases, respondents 
remarked that although they did not know the standards, they were aware that the afterschool 
curriculum that they employed was tied to them. 

Similarly, a few of the program staff were able to give detailed explanations of how 
standards-based contents were integrated into their afterschool curriculum. One program staff 
spoke of the ways in which she embedded standards into her instruction: 

What we do is we break down the standards and make sure we fine-tune every standard. I 
just make sure that every word [of] what I’m teaching is a standard. I look in my book, 
and if the standard is not taught fully, I go to a different text or resource that will meet 
that standard. 

whereas other program staff had more general views of standards-based curriculum, for 
example, a site coordinator at a math program had a more general idea of how standards were 
addressed by their afterschool curriculum: 
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We mainly come up with [curriculum] from general standards of what students should be 
working on, when they are in kindergarten or when they are in first grade. We take that 
information and a combination of what they are working on during the school day. So, 
we get information from the day-school teachers whether it’s weekly or if it’s monthly, 

“My students are working on X, Y and Z this month in math.” Then [afterschool] 
teachers create fun games and activities like that during that time. So, they might have a 
bingo game that’s multiplication or some other thing. 

The survey responses supported the qualitative findings. A different rating scale was 
used in Year 1 for reading and math programs; the staff were asked to rate their familiarity 
with state standards on a scale of 1 to 10 whereas the science, art, and technology programs 
were asked to rate their familiarity with state standards on a scale of 1 to 5. A higher score 
indicates greater familiarity with state standards. In general, it appears that math site 
coordinators were the most familiar, and technology staff were the least familiar. Both Tables 
10 and 11 display afterschool staff ratings of their familiarity with state standards in their 
focus content area. Because there are no homework content standards, homework programs 
were not included for this item. 

Table 10 



Mean Rating of Familiarity with State Standards 





Site coordinators 




Programs 


Scale 1-10 


Program staff 


Reading programs 


8.79 


8.30 


Math programs 


9.5 


8.24 



Table 1 1 

Mean Rating of Familiarity with State Standards 



Programs 


Site coordinators 
Scale 1-5 


Program staff 


Science programs 


4.25 


4.38 


Art programs 


N/A 


3.95 


Technology programs 


N/A 


3.13 



For the technology programs, the survey results indicated a moderate level of 
familiarity with the technology standards. However, a common theme emerged; although 
most staff rated their own knowledge of technology standards at the weak to moderate level, 
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the majority of interviewees (technology staff) went on to describe activities or methodology 
they employed that was clearly reflective of national and/or state standards for practice. For 
example, the majority of programs reported that students regularly engaged in activities 
aligned with the standards-based criteria such as: using technology tools for learning, 
productions, and creative exercises; using multiple technology resources to communicate 
information and ideas; using technology for research; and using technology for problem- 
solving in the real world. 

Link to standards. In the surveys, afterschool staff were asked whether they employ 
standards-based content instruction in their practices. Results revealed that many staff 
included standard-based content in their curriculum. However, within each content area, 
some standards were used more often than others. 

In the content area of reading, more staff reported using lower grade-level standards as 
represented by “Vocabulary development” (96%) and “Reading silently or aloud with 
fluency” (96%). The upper grade-level standards as represented by “Applying self-correcting 
strategies to decode text” (55%) and “Understanding literary techniques” (61%) were used by 
a lower percentage of staff. 

Similar to reading, math also reported applying lower grade level standards such as 
“Addition and subtraction of whole numbers” and “Pattern recognition” more often, only 
50% of the math staff reported on applying higher level standards such as “Problem solving 
using equations” and “Understanding/applying mean, range, and median.” 

Most of the arts staff reported employing standards-based arts curriculum. The most 
commonly applied (93%) standard was “apply art techniques and processes.” The least 
common practice was in the integration of arts with technology, where only 33% of the arts 
staff indicated that they “use technology as a creative tool.” 

In general, most science instructors indicated employing standards-based curriculum. 
Due to the vast numbers of contents available, there were wide variations in subject 
selections. For example, only 33% indicated that they worked with “reproduction and 
heredity” in their instruction, whereas 80% indicated that they used the content 
“understanding of science/technology.” 

For technology, most of the program staff indicating that they gave students 
opportunities to “practice responsible behavior and use of technology.” The least visited 
standard which only 43% of technology staff reported applying was “research and evaluate 
the accuracy and bias of electronic information.” Once again, grade level would play a role in 
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the staffs selections of content topics, with most of the choices leaning towards the lower 
grade levels. 

Because there were no homework content standards, homework instructors were not 
included. The details of each standard-based content and the related percentage of usages are 
found in Appendix A. 

Linkage to School 

According to Clayton (2004), “Afterschool programs need a strong connection to the 
learning objectives of the day school in order to increase student achievement.” This 
continuity of learning between the school and afterschool program is supported theoretically 
in the work of Noam, Biancarosa, and Dechausay (2002). These researchers posit that the 
“bridging” of school and afterschool helps to promote more meaningful academic learning. 
Miller (1995) further emphasize that it is possible for students to “increase [their] sense of 
themselves as learners” and to “transfer positive experiences in a school-based afterschool 
program to more positive feelings about school itself’ (p.46). The pattern of communications 
and relations of the day school and afterschool program were examined. This section will 
present results on the frequency of communication between afterschool and day-school staff 
as well as links to the day-school curriculum. 

Frequency of communication. The staff survey inquired how frequently the 
afterschool staff communicated with the day-school teachers. Figure 11 displays the reported 
frequency of communication. 




Figure 11. Frequency of communication between afterschool program and day-school staff. 
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As shown earlier, slightly less than half of the afterschool staff indicated that they 
communicated with the day school frequently or regularly, while an equal number of 
afterschool staff indicated that they never or rarely communicate with day-school staff. 
About 20% of afterschool staff reported that they sometimes communicated with day-school 
teachers. 

Links to day-school curriculum. Because a critical component of effective afterschool 
programming is the coordination of program content to the day-school curriculum, the 
interviews conducted with the afterschool staff probed deeper into the connections between 
day school and afterschool. A number of distinct themes emerged from the qualitative 
analyses. 

Project directors, site coordinators, and program staff were requested to describe the 
styles of communication and coordination between the day school and afterschool. With a 
few exceptions, most afterschool staff reported a more casual form of communication with 
the day-school teachers, typically on an “as-needed” basis in regards to individual student 
progress and/or behavioral issues. Communications with the day-school teachers occurred 
mostly in forms of brief, informal contacts such as email or casual drop-in conversations. 

A few programs described a more intentional approach. For example, overlap in day 
school and afterschool staff was mentioned as a strategy to allow for more collaboration and 
communication between day school and afterschool. In some district-affiliated programs, 
they referred to this process as a “seamless approach.” In other instances, a YMCA program 
made use of agendas, planners, and homework logs to communicate with day-school teachers 
relating to homework assignments, and coordination of curricular contents. Overall, the key 
topics on most of the communication between day school and afterschool were homework 
related, indicating homework as a crucial component of most academic activities in 
afterschool and a means of connection between the day-school and afterschool teachers. 

An interesting exception was the science afterschool programs, where the majority of 
site coordinators (90%) and staff (93%), reported that specific staff members were assigned 
or responsible for communication with the day school. At a closer look, six of the nine 
science programs are district affiliated, with two of the other three community-based 
programs indicating strong ties with the day school. 

Some general commonalities were also found in terms of adjusting practice according 
to student need. For example, most district affiliated programs reported including input from 
their day-school teachers when fine-tuning their curriculum. Other programs also solicited 
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input from parents and students; two programs actually employed an Education Coordinator 
in their programs to manage coordination with the day school in curriculum development. 

Exemplary Practices: A Collaborative Approach 

Staff at one program in particular described a science program that was clearly the 
result of a collaborative effort between day-school and afterschool instructors, designed 
around and coordinated with the district’s science curriculum, grounded in state and/or 
national science standards, and supplemented with purchased science kit materials. Science 
lessons revolved around a fish hatchery theme, developed primarily during day-school 
instruction, and maintained and studied throughout the year in both the day school and the 
afterschool program. According to interviewees, this concept provided a focus that gave 
students a beneficial sense of continuity between day school and afterschool science 
instruction. One site coordinator explained, 

What’s really neat about this is that it was a collaborative project.... We worked together. 

Not just one school building, but we got together and partnered with another building and 
we made sure that we had representatives from across the grade levels. So we’ve got 
pre-K through fifth grade and then specialists represented, and we designed the 
curriculum.... We went ahead and tied this curriculum to the national science standards, 
which will make it very easy for our day-school teachers to incorporate this program into 
the day school, because it’s electronic. All of their lesson plans are ready to go for this 
program, and it will all be tied to the national standards. 

Research-based Practices 

Afterschool program instructors were asked about the use of research-based practices 
and teaching strategies. This section will explore the curricular design and decision making 
structures, as well as content specific instructional strategies. 

Curricular design. Most afterschool programs indicated that they designed the 
curriculum, rather than rely on pre-designed or purchased curriculum. At one arts program, 
decisions about the arts curriculum were clearly a collaborative effort in terms of both 
afterschool teamwork and working closely with community partners and the day school. This 
program’s approach appeared to be integrative in many respects, but most evidently and 
effectively with regard to arts curriculum development. The arts community partnership 
director at the afterschool program explained the process in detail: 

[Our arts program is] self-designed. We don’t purchase any programs.... We don’t 
purchase a curriculum. We work with a variety of artists.... We do rely on my knowledge 
and our artists’ knowledge.... We are really responsive to the community and what we 
understand their needs to be at this moment. We create curriculum that addresses that. 
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We do that together. The artists ultimately are the ones that carry that out. They’re the 
ones working with the kids on a day-to-day basis. We talk together with them. [The 
project director] and I, we meet with the principals and different representatives before 
we do any scheduling for the session, to talk about the curriculum areas that are in need 
at this time. We take those ideas and talk to our artists, and together come up with the 
format of what we're going to do. The artists do the lesson plan of the actual lesson. [It’s] 
a joint process between [the project director], myself, our artists, and the schools. We 
meet with the principals. We start with the themes or the curriculum areas that need to be 
addressed.... Then we look at each session and what an overarching curriculum area 
might be for that session.... Then we work with our artists to come up with a variety of 
ways to address those topic areas.... In a variety of art mediums, we’re addressing [those] 
topic area[s]. 

This approach to curriculum design was also echoed by a site coordinator at a science 
afterschool program: “[The curriculum is] self-designed — nothing is cookie cutter 
here.... Our teachers design everything that we do here,” while another site coordinator stated 
simply that the program staff “come up with their own curriculum and they make sure it ties 
with the [state] standards.” 

On the other hand, some programs did indicate using curriculum from the day school or 
that were pre-designed. The project director at another science afterschool program indicated 
that they relied primarily on the day-school curriculum: 

First of all, we use what is being taught in the day school. We take whatever curriculum 
they’re using and enhance that. The way that we do that is we actually use our regular 
day teachers as our afterschool teachers. The curriculum is already there. We use what is 
on their lesson plans. We take from that and work with that as far as what we do in the 
academics. We also advance into that by having projects and programs with research 
based things, such as Design It, which is a science project. We use their curriculum for 
our science activities, Wide Span for reading and math. 

Similarly, a site coordinator at a reading afterschool program remarked that the 
curriculum used was pre-designed and straightforward: “It’s very recipe-like. It tells them 
exactly what questions to ask prior to the book, the activity suggestions. It’s very laid out for 
them, so it’s very user-friendly.” 

Overall, whereas some programs chose to use pre-designed curriculum and others 
preferred to design their own, all programs had consensus that they want to employ 
curriculum that would support students’ academic competence, while making learning fun 
and less rigid than in the school day. 
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Content Specific Instructional Strategies 



From extensive literature reviews, research-based practices in the six content areas 
were extracted. Some of the common practices across the six contents included scaffolding 
techniques that appeared to be correlated with grouping strategies, and the “real world” 
approach that was used to encourage student learning by infusing the learning process with 
cultural and social experiences that were relevant to their lives. Other common practices in 
science, arts, and technology included a “hands on” approach; active participation, “learning 
through doing,” students making connections on their own, and the constant shifting from 
guided work, to group work, to pairs work, then to self-paced work, and back once more to 
guided work at a higher level. 

In general, survey data suggested that afterschool staff employed a wide range of 
research-based practices to support and facilitate their students’ learning. The analysis of the 
survey data also revealed that there were variations based on the content focus. For example, 
technology programs reported the most frequent use of research-based practices whereas 
reading programs reported the lowest frequency. Furthermore, technology, science, arts, and 
homework programs appeared to be more focused on developing higher order thinking skills, 
whereas reading and math programs were more focused on direct teaching. For instance, one 
technology instructor talked about teaching her students to problem solve: “You have to 
know how to fix it. How do you solve this problem? The engineer is sitting in that seat now, 
and something goes wrong, you’ve got to troubleshoot. That’s part of life with anything. 
Troubleshooting and problem solving skills are needed.” Specifically, in science content, the 
most frequent practices were (a) describing scientific procedures to students, (b) using tools 
to gather and analyze data, (c) designing a scientific investigation, (d) conduct simple 
experiments, and (e) using evidence to predict and explain. All these strategies were designed 
to promote applicable techniques and higher order thinking skills and were practiced fairly 
evenly across all science programs. 

On the other hand, math and reading programs were more focused on direct instruction. 
For example, math staff reported the use of mathematical tools, asking students to solve 
world problems, and providing instructions to solve math problems most frequently. The less 
frequently practiced instructional strategy appeared to be in the arena of developing students’ 
higher order thinking skills such as providing unstructured opportunities for students to 
investigate and explore, and having students write justifications for their work. The least 
practiced strategy in math and reading programs was having students actively participate in 
hypothesis testing. In many of the math and reading programs, program staff mentioned 
teaching knowledge facts through direct instruction. One math staff commented, “So we have 
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pretty much tried to stay with just basic facts for them to learn math facts.” Another reading 
staff talked about reading instruction: “In particular I work with two students that their 
parents requested them to stay here with the afterschool program because they don’t have the 
support at home to read and learn vocabulary with their parents. So I try to make a point to 
read and study words with them.” These trends can be results of the differences in content 
characteristics, and/or a process of program maturation (since math and reading were visited 
in Year 1, the afterschool program as a field has gone through rapid development and 
maturation). 

For more detailed analyses (afterschool staff were asked to rate the frequency with 
which they employed specific instructional practices on a scale of 1 to 5, with 1 indicating 
never, and 5 indicating more than once a week), the frequencies of the usage of specific 
instructional practices under each content area can be found in Appendix A. 

Cross-Content Integration 

Over the six content areas, the curriculum for technology was the most integrative; 
these programs incorporated technology learning into many of the afterschool activities 
across academic content areas including science, reading, math, and arts. One site 
coordinator commented on the integration of technology into the afterschool curriculum: “In 
most all of our subject areas technology is incorporated somehow and integrated into it.” She 
further expanded: 

I talked about the language arts. The math you could get into how many beats per minute 
for a song. You have to do counting. When someone is doing the background music you 
have to figure out how many counts, how many beats per minute or per stand that they’re 
doing it on. So there's a lot of math involved with that. Science, I’m sure there’s a 
science, but I can’t pick it off the top of my head. Art, we actually have one of the 
teachers from Henninger’s. She incorporates a lot of art, and especially when they do 
famous groups. They might work on a comic book or a stage set, so they do a lot of 
drawing and things like that or even computer parts. 

Homework programs also had a broader conception of instruction that focused not only 
on homework assistance and completion, but also on developing study skills, work habits, 
and organization skills. The four skills most staff mentioned enforcing were: development of 
time management skills, development of skills in using reference material, development of 
note taking skills, and strategies for test preparation. For instance, one homework staff 
member commented on modeling note-taking for her students: “I’ll give notes to try to 
explain to the kids this is how you should take notes when I’m doing it. You don’t have to 
write everything down, but then there are some teachers that want you to make sure you 
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write everything down.” Another staff member mentioned teaching study skills as an 
important component of afterschool homework instruction: “Teachers sort of become experts 
at homework at some point. I do work with my students on study skills. It’s really what I 
focus on. It doesn’t matter if it’s homework or if it’s class work or if it’s just some 
extracurricular activity that they’re working on, but I really work on study skills with them.” 

Summary 

In general, these programs adhered to the structure of goal-oriented programs as 
reviewed in the literatures. Almost all programs under study had set clear program goals and 
objectives, and designed strategic plans to achieve their goals. For example, some of the 
math, reading, and homework programs had improvement on academic achievement as their 
program goal. These programs hired credential teachers as instructors, focused their 
curriculum and teaching strategies to enhance student academic achievement and academic 
skill building. 

Math and reading programs tended to focus more on improving basic skills whereas 
science, arts, and technology programs had goals of exposing and engaging students to their 
specific contents. The science, arts and technology programs hired and/or collaborated with 
experts in their field, and emphasized exposing students to a variety of experiences in their 
field, provided students with enrichments and personal experiences, and tended to focus on 
developing the higher-order thinking skills such as gathering data, analyzing data, and 
reporting/presenting the findings. Homework help programs also intended to provide more 
skill building to their students through improving students’ organization, time management, 
and test preparation skills in addition to supervising and monitoring homework. 

At the same time, all 53 programs had broad knowledge of National and State 
Standards in their content areas, and alignment with standards was quite obvious at most 
sites. However, although most programs were employing standard-based curriculum, some of 
the program staff were not well versed in expressing links between standards and their 
curriculum. As a point of interest, while almost all programs adhered to standard-based 
curriculum, most of the programs, regardless of grade levels, tended to focus more on the 
basic standards rather than the more advanced standards in their content area. This may be a 
reflection of the demographics of the student population because all the programs were 
working with low-performing schools, and perhaps populated with more primary grade 
students. 

Some of the common research-based practices across the programs included the “real 
world” approach that was used to encourage students’ learning by infusing the learning 



57 




process with cultural and social experiences that were relevant to their lives as well as “hands 
on” approaches in science, arts, and technology; students given the freedom to test and 
expand their own knowledge through doing, stressed on students “learning through doing” 
and to make connections on their own, and the constant shifting from guided work to group 
work to pairs work, then to self-paced work and back once more to guided work at a higher 
level. 

Although all programs appeared to maintain positive relations with the day school, 
most of the linkages were in casual formats and communication with day-school teachers 
often happened on an “as needed” basis. In these occasions, homework was usually the key 
topic. Some programs, especially the district-run programs which usually constituted shared 
staff, had better communication systems in place, whereas a few district-run programs 
emphasized a “seamless approach” where afterschool program could be considered as an 
extension of the school day. A few community-based programs also made specific efforts to 
maintain open channels between school and afterschool. In one program, a key person was 
designated to liaison between the day and afterschool. In another program, a system in form 
of a homework-log, check list, and meeting dates was set to ensure frequent correspondence. 

EVALUATIVE STRUCTURES AND PROGRAM IMPACT 
Program Evaluation 

Nationally and across a number of states, there is a growing emphasis on the evaluation 
of afterschool programs. Funders want to know that their investment is making a difference 
and are encouraging programs to engage in continuous improvement, making assessment and 
evaluation an integral part of their functions. 

There are typically two types of evaluation that apply to afterschool programming: 
internal and external. Internal evaluation is a process of quality review undertaken within an 
institution for its own ends. External evaluation is the appraisal process performed by an 
agency or individuals not directly involved in or responsible for the program or activities 
evaluated. Interview and survey responses across the 53 participating programs in this study 
indicated that even though rigorous examinations of data are very rare, all of the afterschool 
programs appeared to be conducting varying degrees of internal and external evaluations of 
their programs. 

Overall, project directors seemed to be the most knowledgeable about evaluation, 
particularly external evaluations. Site coordinators were usually familiar with their individual 
sites’ internal evaluations and quality assurance procedures. Program staff were most familiar 
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with informal student assessment procedures and tools utilized to monitor student progress at 
the classroom level. 

External Evaluation 

Responses from interviewees suggested that many of the programs were evaluated 
externally, sometimes by an evaluation organization experienced in program evaluation. 
Most interviewees from all programs consistently indicated that most of the evaluations were 
of the entire program. The methodologies utilized in these evaluations typically included 
pre-post testing or classroom evaluations, comparison groups, surveys, focus groups, 
observational assessments, or a combination of these methods. In general, interviewees from 
the majority of the programs reported positive results from these evaluations, although 
specific results were not supplemented and substantiated with reports. 

Only the technology programs reported having content specific evaluations. These 
evaluations were undertaken by a variety of public and private organizations including Dell, 
Intel, and several universities. The evaluation tracked the overall effectiveness of the 
technology and larger curriculum, often using computer software to monitor student progress 
in the day school and use that as a barometer to adjust afterschool programming. 

For the reading and math programs that were closely affiliated with the school districts, 
over half did not conduct an external evaluation; and approximately one third of programs 
mentioned having an external evaluator. Responses to questions regarding program 
evaluation also varied in the interview data, most likely attributable to different 
interpretations of the term “program evaluation.” A significant number of the programs 
considered the process of preparation for the Annual Performance Report (APR/ PPICs) and 
the visits from the state coordinators as a form of external evaluation. 

Internal Evaluation 

All but two programs reported having conducted some formal or informal internal 
evaluation. Evaluation varied from informal conversations between afterschool staff, day- 
school staff, and parents to a formal administration of surveys to students, parents, staff, and 
tracking of test scores, grades, and attendance records. One program reported a relatively 
rigorous formal evaluation process that involved quarterly assessment reports from staff, pre- 
post testing, tracking of grades and student progress in the day school, and student surveys on 
program satisfaction. The program director explained the intent of the evaluation: 

Once a quarter, each [site] has to come together and present to the Board as well as [the 

director] about what happened last quarter. It's not just what happened from a laundry 

list. We’re trying to create a learning organization. It’s an opportunity for the unit 
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directors to share among themselves and to learn from each other. It creates opportunities 
to share ideas. And it bring a little bit of accountability in. 

Other types of informal evaluations attempted to capture program effectiveness in all 
areas. This included staff meetings as a space for discussion about what was and wasn’t 
working, talking to students, parents and day-school teachers, collaboration among peers and 
anecdotal evidence from individual students who had transformed during or after 
participation in the program. One instructor explained, “There is an informal evaluation 
where we’re always talking about what’s working and what doesn’t work and about what we 
could do differently.” At another setting, a project director said that questions such as “Do 
you feel [the program is] effective? Do you think your child has learned?” were addressed to 
parents and teachers to help the program in self-assessment. 

In general, interviewees most frequently mentioned using the results of internal 
evaluation to serve as a baseline for instruction, monitor student progress, and to document 
program impact. For example, the principal at one of the science afterschool programs 
reported that student achievement data from an internal evaluation were used to revise 
science curricula at the afterschool program in order to align instruction with the standards 
and improve student performance: 

That was the reason why the [afterschool program] curriculum went under revision, 
because [our students] were not measuring up nationally. When you look at their 
assessments on the [state test]... they weren’t measuring up with other states; and because 
of that we had to go back and revisit our curriculum to see where we were falling through 
the cracks. What we found was that we [needed to revise] our objectives — they were too 
narrow. So teachers were limiting instruction to just those methods, when really there 
was a whole standard that needed to be scored. There was a whole area that children 
needed to know by the end of the school year, so that revision had to take place. 

Interestingly, a pattern emerged in the resources and purposes of evaluations between 
the community-based programs and the district-affiliated programs. It appeared that the 
district-affiliated programs seemed to have easier access to student academic databases, and 
about half of these programs made use of this access to establish baselines and determine 
areas of academic need for their students. The student academic database and classroom 
grades were also used as outcome measures in determining academic growth and progress, 
especially for programs with the explicit goal of raising students’ achievement scores. 

The community-based programs all reported conducting internal evaluations on a 
regular basis, some more intensive and extensive than others. They appeared to have a 
broader focus, and utilized a mixture of parent, staff, students’, and classroom teachers’ 
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surveys to seek feedback. These data served to inform programs from multiple perspectives, 
provided a continuous feedback loop for self-improvement, and measured program 
satisfaction rates. 

Perceptions of Program Impact 

All programs reported having positive impacts on their students in some ways. Most of 
the evidence provided was anecdotal, but a few formal evaluations and systematical tracking 
of student progress showed improvements in attendance, classroom grades, and achievement 
scores. At the less quantifiable level, several programs reported improvements in attitudes 
toward schooling, student efficacy, confidence, and engagement. Moreover, individual 
success stories highlighted the transformational potential of the afterschool setting in 
improving not only study skills, attitudes, and behavior, but also increasing the self-esteem of 
students and their social competencies. 

Staff 

Most staff interviewed reported that their program had impacted students positively in 
some way. Although most feedback on this topic was anecdotal, interviewees most 
consistently cited improvements in the areas of attendance (both in the day school and the 
afterschool program), engagement and interest in the content area, social behavior, and 
student achievement. 

Interviewees indicated that their students’ attendance, both in the afterschool program 
and in the day school, was consistent and indicated growth over time. However, almost none 
of the interviewees provided evidence linking attendance to achievement. Most of the 
programs also noted that there was a waiting list for their programs, and indicated that there 
were no recruitment or retention concerns. Although most respondents mentioned that 
students loved to come to the afterschool program and even preferred attending the 
afterschool program to the day-school program, very few provided enough detail to draw 
substantial conclusions. Nonetheless, afterschool program staff perceptions on program 
impact are worth noting here, and a good number of staff members commented that student 
interest in their afterschool program had facilitated improvements in attendance. One arts 
instructor noted, 

Attendance is unbelievable. I have kids that say, ‘I only came to school today because 1 
knew I was going to be working with you.' I feel, just from talking with my teachers, that 
behavior problems in some instances are resolved. Students have success in my class. 
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Furthermore, the following impressions offered by the project director for one of the 
science afterschool programs indicate a breadth of afterschool program impact not always 
captured by quantitative data: 

I want [the program] to be measured by supporting school success, because that is what I 
think leads to academic achievement. We have clear research that says that the children 
in our afterschool program have better attendance. Their truancy has decreased 
significantly. They’re not tardy. Their behavior in class increases significantly. Their 
connection to a professional adult as measured by the child increases significantly — 
statistically significantly.... Those are the things to me that eventually translate into 
school success. So we do think we're improving academic achievement.... We think that 
is significant. 

A technology program director reinforced this assertion and explained: 

There's a whole part of afterschool that is so hard to put into an evaluation form. That’s 
how you change lives and what you offer, the possibilities and dreams that can come 
true. That’s so hard to put into a question. 1 think it’s very important that everybody 
understand the power that can come from an afterschool program in affecting change, not 
only cultural change on campus but also individually in particular lives. I have seen a 
whole campus turn around demonstrating that learning can be fun and can still address 
the needs for standardized tests. Their sense of accomplishment that comes from being in 
one of our programs where they have more freedom to explore and have more hands-on 
experiences is profound. 

Some program staff also mentioned improvements in students’ achievement and 
attitudes towards learning. Most of these feedbacks were clearly anecdotal and related 
directly to staff perceptions. Yet, some programs were able to cite specific academic gains; 
for example, some of the district-affiliated math and reading programs have observed growth 
in district test scores and/or classroom grades, and many homework staff stated that the 
students’ achievement was greatly related to the homework assistance given at afterschool 
programs. This was corroborated by a day-school principal, 

Our test scores have risen every year for 4 years. They had gone down every year for 7 
years before [the homework afterschool program stalled at the school]. In the last 4 years, 
and including this year again, they’re up anywhere from 3-5%. A lot of it is because we 
started working together. 

Incidentally, all program staff across the content areas mentioned a marked 
improvement in students’ engagement and interest in the content area over time. Almost all 
interviewees across the 53 programs reported that students were happy and enthusiastic to be 
at the afterschool programs. They were eager to attend the programs, and some staff reported 
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that even when the students were absent during the school day, they still wanted to come and 
attended the program. Additionally, staff also frequently remarked that students expressed a 
new appreciation for, and interest in the focused content area. For example, one science staff 
member proudly explained, “I’ve had my seventh and eighth graders tell me that they want to 
go into a career in science instead of being a dancer or a football player or something like 
that. They’ve actually told me they want to get a career in science.” Similarly, a staff member 
of a technology program explained the value and relevance of her technology instruction, 

We see a big improvement at the high school. What we're finding is if you give the 
student the laptop, which we provide for every student to take home and use, they 
complete their homework and it’s higher quality.... Even our students that have social 
problems, when they're more fixed on a project that [uses] the computer, they do a lot 
better. We see a lot of better behavior in those children. 

Consistently, all program staff reported having positive relationships with their 
students, and suggested that the afterschool programs have had positive impacts on students’ 
self-confidence, social skills, and attitudes towards learning. Observation data supported the 
interview data, with students observed to be actively engaged in most of the sites visited. 
Programs appeared to make specific efforts to select materials that were of high interest to 
students, and program staff engaged students with meaningful questions and frequent 
attempts to get students involved. A majority of the interviewees indicated that their students 
enjoyed and benefited from the afterschool program, and some felt that the afterschool 
programs appealed especially to students who were academically disengaged by offering 
them the kind of personal attention and assistance they needed. 

Parents 

Parents were surveyed and their inputs on program satisfaction and perceived academic 
impacts were analyzed. In general, parents reported seeing improvements in their children’s 
skills and overall interest in the specific content area, their interest in school work, and in 
finishing the specific content area homework on time. Parents were also inquired about 
perceived changes in the students’ skills and interests. Again, survey results indicated that 
parents perceived improvements in the students’ skills and interest after they participated in 
the programs. Figure 12 illustrates the results. 
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□Decreased 

■No change 

□Improved a little 
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□Not sure 



Subject- specific skills Interest in subject Interest in his/her schoolwork in 

general 

Figure 12. Parent perception of changes in student behavior. 

About 75% of the parents perceived that their children’s skills and interest improved 
since participating in the afterschool program, whereas a minimal number of parents (1%) 
saw a decrease. Approximately 11% to 13% of the parents did not perceive any changes in 
their children’s performance or attitudes, while an average of 7% were not sure about any 
changes. The survey also inquired parents on changes in their children’s interest in 
schoolwork in general, and results found that 44% of the parents saw a great improvement. 

Day-school Teachers 

For Years 2 and 3, day-school teachers were included in the survey administrations. 
The first item on the teacher survey determined if teachers knew which of their students 
participated in the afterschool program. Only those who answered “yes” were requested to 
proceed with two follow-up questions in regards to perceived changes in these students since 
they began participation in the afterschool program. Day-school teachers were inquired about 
their perceptions on any behavioral changes on a 5-point scale, from 1 ( strongly disagree ) to 
5 ( strongly agree ) and to provide their assessments of the changes in students’ performance 
and attitudes from 1 ( decreased gready ) to 5 being “( increased greatly). 

Most teachers for the 35 programs in Year 2 and 3 reported that they had noticed 
positive changes in students’ behaviors in terms of school attendance, frequency of 
classroom participations, effort on school work, paying attention in class, and discipline 
problems, their responses are displayed in Figure 13. 
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Figure 13. Teacher perception of changes in student behavior. 



As shown in Figure 13, about 10% of the day-school teachers had not noticed a positive 
change in students’ behavior in terms of school attendance, frequency of classroom 
participation, effort on school work, paying attention in class, and discipline problems. About 
half agreed that their students in these afterschool programs had improved in school 
attendance, were paying more attention in class, and had fewer discipline problems. In 
addition, 58% of teachers agreed that the afterschool program students increased their 
frequency of class participation, and most teachers (61%) agreed that their students in the 
afterschool program put more effort into school work. 

Day-school teachers were also surveyed in regards to their perception of any academic 
performance and attitude changes in students participating in these afterschool programs. 
Academic performance and attitude changes are defined differently according to the specific 
content area. The definitions for the four content areas are as follows: 

Arts: willingness to participate in various arts activities, the overall quality of their art 
projects, perceptual and expressive skills, technical skills (e.g., ability to paint, sing, 
play a musical instrument), attitudes toward the arts in general, and ability to engage in 
higher order thinking. 

Science: science skills, science test grades, attitudes toward science and homework, 
and the quality and timely completion of science homework. 

Technology: participation in technology activities, overall technical skills, attitude 
towards technology, engage in higher order thinking, cooperative collaborative skills, 
persistence in school work, and problem-solving skills. 
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Homework: motivation for completing homework, ability to complete homework, time 
management skills, organizational skills, cooperative and collaborative skills, 
performance on tests, persistence in school work, problem-solving skills, study and 
research skills, and quality of work overall. 

Most notably in the four content areas, a majority of teachers perceived an 
improvement in their students’ overall interest and skills in the content- specific area. Most 
teachers also noticed a positive increase in their attitudes towards the content area. Perhaps 
due to the fact that some of the teachers surveyed did not directly supervise the students in 
arts, science, or technology, up to 30% of the teachers responded that they did not know 
whether there had been a change in the students’ academic performance. Meanwhile, of the 
teachers whose students participated in homework assistance afterschool programs, 47% to 
75% perceived improvements in their students’ performance on tests and their ability to 
complete homework. 

Summary 

In summary, external evaluations on the content- specific component of the programs 
were not always conducted. Only technology programs consistently reported having external 
evaluations on the technology component of their programs. In other programs, if external 
programs were conducted, they were usually program-wide where the findings were 
generally positive and strong evidences to support the claims were rare. Most of the 
programs conducted internal evaluations, some more formal than others. The purposes of 
these evaluations were for program improvements, such as monitoring student progress, 
measuring program growth, research for program needs, and gauging program impacts. 
Except for some of the district-affiliated math and reading programs, student assessments 
throughout the programs were informal, generally in the forms of spot checking, questioning, 
and observing student understandings. 

Overall, programs were perceived by parents, staff, and teachers to have positive 
impacts on student engagement, attitudes, skills, and their knowledge improvements. More 
specifically, programs were perceived to improve attendance, especially if the students were 
engaged in the afterschool programs. Students were happy and enthusiastic to be attending 
the program, and engaged in program activities. All afterschool staff reported having positive 
relationships with their students, and consistently suggested that the programs had positive 
impacts on students’ self-confidence, social skills, and attitudes towards learning. Most of the 
program staff mentioned an improvement in students’ achievement and attitudes towards 
learning. Day-school teachers reported that they perceived improvements in the students’ 
attitudes and skills toward four of the six content areas. Behaviors (i.e., school attendance, 
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frequency of class participation, effort on school work, paying attention in class, fewer 
discipline issues) were also perceived to have greatly improved. In all of the content areas 
(except homework), parents also perceived improvements in content- specific skills, interest 
in a specific content area, interest in schoolwork in general, and finishing content-specific 
homework on time. Parents with children in the homework help programs saw improvement 
in their child’s motivation to complete homework, ability to complete homework, time 
management skills, organizational skills, and interest in school work in general. 

IMPLICATIONS AND CONCLUSION 

The findings in this study indicate that the characteristics of the identified afterschool 
programs appeared to be aligned with the criteria established for exemplary practices in 
terms of program structure, student engagement, instructional strategies, and issues of 
accountability. 

The common characteristics that were found across the afterschool programs are listed 

here: 

• These programs had strong leadership and established clear goals. 

• They aligned program structures and contents to meet those goals. 

• They set a schedule that allowed time for students to leam and practice. 

• They established relationships with the day school. 

• Their curriculum in general reflected a linkage to standards. 

• Most of the programs used research-based strategies. 

• They all maintained some form of evaluative structures. 

• Staff members had low turn over rates. 

• Staff members related well to the students. 

• The staff were able to build rapport, maintained high expectations, kept students 
motivated and engaged. 

Most of the 53 programs in this study had set clear goals and structured their programs 
to meet the goals. To increase the students’ academic skills, many of the math, reading, and 
homework programs used a “seamless approach” to connect the day school with afterschool 
and employed certified teachers. To expose students to the experiences of science, arts, and 
technology, programs used hands-on approaches and generally collaborated with 
organizations that are strong in the specific content areas, such as Arts Corps, Girls’ Inc., 
local universities, Dell, Apple, etc. Almost all of the 53 programs maintained good 
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relationships with the day-school personnel. However, few of them had established formal 
communication systems for that purpose. Moreover, across all the programs, staff had 
knowledge of content standards and engaged in content specific research-based instructional 
strategies. To engage students and staff, these programs recruited qualified staff and created 
working environments that built comradeship and inspired support for the programs’ 
missions. As the staff felt respected, supported, autonomous, and confident in their ability to 
reach their students, these programs were able to retain staff and achieve lower turnover rates 
than other programs. In turn, staff and students were able to construct positive relationships 
with each other, characterized by warmth and mutual respect. This supportive relationship 
permits the staff to serve as role models, creating a norm of high expectations, and 
strengthening student behavior, school attendance, work habits, and attitudes towards 
learning. At the same time, most programs also recognized the importance of considering 
student voices when making decisions regarding program activities and content; hence, many 
programs allowed students to provide input, especially in the arts and technology programs. 
Many of these programs also made use of research-based strategies to engage and motivate 
students, making learning fun. Overall, programs reported some form of internal and/or 
external evaluations for accountability and self-improvement purposes. As a result, students 
were engaged and excited to be in the programs. In general, day-school teachers, parents, and 
afterschool staff perceived the programs to have positive influences on the students. Some 
programs reported increased school attendance, better test scores, and most of the programs 
reported positive changes in behaviors and attitudes towards learning. Parents were very 
satisfied with the programs both in terms of positive changes in their children’s behaviors 
and attitudes, and in general program functioning. 

As discussed earlier, the single most dominant characteristic that is consistently 
revealed in all programs visited was the high motivation level of the sites. These high 
functioning programs were characterized by strong leadership. Staff members were 
empowered to engage in program missions that included internal motivations, such as 
making a difference in the lives of their students. It appeared that this intrinsic motivation for 
staff members to bond with the students was a powerful force toward the aspirations of both 
the staff and the students. This relationship can be further enhanced, so that the afterschool 
staff can foster the relationship to induce academic resiliency in their students. Implications 
focused on these areas will be discussed. 

Implications 

As the quality of afterschool programs is further embedded in the social systems of 
schools, families, communities, networks, state, and federal policies and regulations, in order 
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to induce sustainable change in students, all the systems have to function collaboratively 
together. Because the key findings of this study point to the strength of strong leadership and 
the motivational value of the staff- student relationship, there is the potential of enhancing 
their skills to create a tighter social system and bring in more resources from the day school, 
families, and communities. In order to do that, there is the need of providing meaningful, 
informative, and regular professional development opportunities to the afterschool staff to 
further enhance their abilities and collaborative skills. However, the findings of this study 
also indicate that the participation rate in professional development is quite low. Thus, 
additional incentives through policies and guidelines are needed to encourage staff to take 
part in the professional development. These can be achieved through: (a) policies that support 
staff (professional) development, (b) funding sources and designations, and (c) ongoing 
program evaluation to prescribe professional development needs. 

Professional Development 

The study’s findings show that professional development was not regularly offered at 
all programs, and when offered, participation rates tended to be low. Moreover, project 
directors and site coordinators appeared to attend conferences and workshops more often 
than the program staff. This can be due to many possible reasons: the higher turnover rate of 
other staff members (in comparison to project directors and site coordinators), the tight 
scheduling in afterschool programs (which makes it difficult to spare staff members for 
professional development), and limited budgets to pay staff time for professional 
development or to pay for substitutes. Because professional development has a strong 
correlation with staff efficacy, quality of the delivery of the intended lessons, and hence 
student outcomes (Ingvarson, Meiers, & Beavis, 2005), federal and state policies guiding the 
development and operations of afterschool programs should take the study findings into 
consideration and provide additional guidelines and regulations for the provision of sufficient 
quality professional development opportunities for afterschool programs. 

At the site level, program directors and site coordinators should provide professional 
development opportunities to increase staff readiness to address student needs. Program 
directors should re-examine program goals to decide whether site coordinators or program 
staff would benefit most from attending certain conferences and professional development 
opportunities, and make decisions accordingly. If there is a “training-of-the-trainer” approach 
at the sites, a systematic system or manual should be established to ensure professional 
development fidelity. When planning the yearly calendar, time off for professional 
development, reserving substitutes for the staff to attend professional development, or 
closing the program a few times a year to allow for staff development should be considered. 
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The findings of this study pointed to several specific areas where afterschool staff 
would benefit from professional development: (a) content and curriculum, (b) collaboration 
with the day school, (c) evaluation and assessment, and (d) parent involvement. Each of these 
areas of professional development will be discussed. 

Content and curriculum. The study findings showed that although most program staff 
were aware of the standards within specific content areas, they were less knowledgeable 
about the links between the standards and their instructional practices. To support program 
staff in becoming more familiar with curriculum standards and to be more creative with the 
implementation of these standards in their instruction, professional development in content 
and curriculum should be consistently provided. 

While discussing relationship building and curriculum content and standards, it should 
be noted here that the positive relationships existing among the students and staff might be 
due to a variety of unexplored variables (e.g., that the afterschool staff were from the same 
community; the afterschool staff shared similar life experiences as the students and were able 
to connect to them better; a certain aspect of their personality made them more appealing and 
trustworthy to the students, etc.) that researchers are not yet able to affirmatively identify. 
Thus, one should not be hasty and take the short route by in replacing current staff with 
certified staff in order to promote better academic outcomes, instead professional 
development and enhancing the readiness and knowledge base of the current staff would be 
the recommended route. 

To facilitate this practice, state agencies should set policies to guide program 
functioning and provide periodic monitoring to ensure that the staff have received adequate 
professional development to meet program expectations for high quality, engaging academic 
instruction. State agencies should also work closely with professional development providers 
and afterschool programs to coordinate the professional development topics offered at 
national and regional conferences to meet staff needs and interests. With the efficiency of 
technology, toolkits such as the ones offered by this Afterschool Partnership make it possible 
to instantly deliver to any program the most updated instructional practices in a cost-effective 
way, and these should be utilized at all program levels. 

Furthermore, when making policies regarding afterschool programs, policy makers 
should keep in mind that even with the best preapproved, flawless research-based 
curriculums, the delivery of the content is as important as the content itself. Thus, policies 
and legislation guiding program certifications and re-certifications should consider the need 
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for staff members and administrators of afterschool programs to maintain continuous 
professional development, updates and support with the new developments in the field. 

Collaboration with the day school. The study findings indicated that the day school 
plays an influential role in the efficiency of their affiliated afterschool programs. The day 
school can provide space (e.g., classrooms, playgrounds), and resources (e.g., use of 
computers, academic and behavioral data on the students, access into classroom records, 
professional development opportunities, etc.) which are important to afterschool program 
organization and student learning. All of the programs visited in this study maintained good 
relationships with the day school. However, as important as this relationship is, very few 
programs in this study had set up strategic systems to establish and strengthen this 
relationship. Professional development can assist by providing professional development in 
setting up procedures to establish channels of communication, and identifying systematic 
approaches for after school programs to make contacts and collaborations (e.g., 
communication logs, planners, email, meetings, and ideas for developing more collaborative 
program structures like overlap in day school and afterschool program staff, offering 
afterschool program staff paid time to communicate and work with day-school staff, and 
administrative support in creating connections to the day school). 

Furthermore, state and local educational agencies can organize professional 
development opportunities for local afterschool programs to get together and share their 
experiences with each other; the state and federal program officers may even want to 
participate in specific professional development for the delivery of specific curricula together 
with their programs, so that they can have a realistic expectation of what the program can 
achieve and help set appropriate benchmarks. 

Evaluation and assessment. As outlined in the logic model, continuous evaluation is 
necessary to gauge program performance and strive for continuous improvement. However, 
across the programs, evaluation was mostly used to gauge program impact and monitor 
student progress, using evaluation as a reflection to guide program improvement was not 
practiced as frequently, especially at the staff level. Professional development could stress 
the importance of this continuous feedback loop for the sustainability of program quality, and 
provide programs with the skills and tools to jump start this learning cycle. First, afterschool 
staff would need to develop strategies on how to locate/develop reliable self-assessment tools 
that can measure program growth and improvement. Once programs have the tools to assess 
program results, professional development should be provided on how to interpret findings 
and translate assessment results to improve afterschool programming and instructional 
practices. In addition, student assessment and tracking of student progress should also be a 
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priority in professional development. Through regular monitoring of student work and 
achievement, afterschool programs can have a better understanding of the needs of their 
students, and areas of support that need to be improved. 

Parent involvement. In terms of parent involvement, although the research literature 
continuously stresses the importance of parent involvement in influencing children’s 
academic outcomes, this study repeatedly found that parents, though very satisfied with the 
programs, were generally not involved in attending events or volunteering in afterschool 
programs. This finding is consistent with other literature on afterschool programs (Huang, et 
al., 2007; Huang, et al., 2006). These results suggest that perhaps perceived parent 
involvement and parent expectations are just as important as actual attendance at events. 
Although some parents may be less able to attend afterschool events due to circumstances 
such as long work hours, other children in the household, being a single parent, and not 
knowing how to be involved; parents may be highly involved in other ways such as giving 
their children access to educational resources (e.g., books, computers), and in emphasizing 
the importance of a good education. Some of the culturally specific ways of being involved 
may be just as important as attending school events especially for immigrant parents and 
parents who feel less efficacious in dealing with the educational system (Zhou & Kim, 2006). 

Despite the structural constraints families conceivably face, professional development 
in the area of parent involvement can focus on training staff to view parent involvement as a 
broad set of behaviors that could involve not only attending school events, communicating 
with the afterschool staff, but also in providing resources to their children for learning, 
holding high expectations and aspirations for their children’s education, and developing and 
employing social networks to gain information and links to educational opportunities. In 
understanding and supporting different styles of parental involvement, staff can build 
partnerships in various forms with parents in supporting the students. Furthermore, 
professional development can also be helpful in providing staff with ideas on how to develop 
rapport with parents, such as focusing on student strengths, maintaining frequent 
communication and contact, offering suggestions and strategies for different forms of 
parental involvement, emphasizing the importance of parent involvement, and giving parents 
a voice in afterschool programming decisions. 

Afterschool Funding 

Another area that can have important influences on the quality of afterschool 
programming is funding sources. In writing funding contracts, funding agencies can include 
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categories in the budgets specified for professional development, staff stability, and 
involvement with the day school and parents. 

Professional development. As mentioned in the previous section, professional 
development is a necessity for afterschool staff. Funding agencies should create specific 
budgets for continuous staff development as a way to ensure that content staff will be 
adequately prepared and continuously updated with new information. If the goal of funders is 
to target professional development for specific staff members such as instructors, site 
coordinators, or coaches, it should be clearly specified in the contracts for afterschool 
programs. When allocating funding for training and professional development, the costs of 
staff time and the cost for the substitutes for the staff members attending professional 
development should be considered together with the overall cost of the training. 

Staff stability. The study results also indicated that at these high functioning programs, 
staff turnover rate was considered to be low. Over 60% of the staff had between 1-7 years of 
experience at the current site and over 30% of the staff had over 4 years of experiences at the 
current site. Staff stability is important for relationship building, because staff members serve 
as constant role models and mentors in their students’ lives and provide the basis for students 
to build trust, positive attitudes and efficacy toward learning. To reduce staff turnover rate in 
the afterschool field, funders and afterschool administrators should consider incentives for 
building morale such as rewards (further educational opportunities such as a grant for tuition) 
to the “outstanding afterschool team/teachers” nominated by parents, teachers, or the 
students. Furthermore, recognition for the outstanding performance of specific staff, a site, or 
program may also offer acknowledgement for high quality performance. A pay scale 
incentive for years of service and a possible career ladder would also help staff retention. 
Policy makers and legislators may consider retention plans that establish policies and 
legislation to recognize program workers’ contributions to student success. 

Involvement from day school and parents. In order to ensure linkages between the 
afterschool programs and day school, some formal agreements should be made prior to the 
inception of the grant/contract writing. For example, the administration at the day school can 
connect to certain conditions and resources, such as granting the use of certain facilities at the 
school, providing time for school teachers and afterschool staff to meet and plan lessons 
together, establishing a system of communication, etc. (e.g., homework log between day- 
school teachers and afterschool staff). Funding agencies can budget additional resources for 
afterschool programs that would facilitate these linkages with school. For instance, the 
budget may include a person designated as the day-school liaison who can establish a chain 
of intentional communication and collaboration with the day school. Additional funding for 
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relationship building such as staff retreats or workshops (together with day-school teachers) 
can also be a way to form collaborative relationships. 

Similarly, to develop relationships and opportunities for parent involvement, funding 
agencies should budget for parent participation, time for staff to communicate and 
collaborate with parents, and resources for families. Funding allocated to providing childcare 
during parent events, (e.g., ESL, parenting, other educational classes, Open Houses, parent- 
teacher conferences) will facilitate parent participation. Home visits and family assistance 
can further solidify the relationships between the afterschool and the families. 

The importance of parent involvement specific to afterschool programs and student 
outcomes should be studied further and more rigorously. If the importance of the 
relationships is confirmed, policies can be set to encourage parents to make certain 
contributions of their time when their children enroll in the program. It should be noted that 
parents’ contribution of time should not always require time spent at the program site; a 
project that the parents work on with their children at home can also be considered as parent 
involvement. 

Continuous Evaluation 

Continuous evaluation is an important component of high quality afterschool programs. 
Implications of our findings in this area suggest a need for more systematic evaluation, 
enforced through local and federal policies and regulations. Specifically, the use of 
evaluation tools in program planning, curriculum development, and general improvements 
can lead to improved student outcomes. This section will discuss implications at the 
classroom level and at the program organization level. 

Classroom assessment. Research literature has consistently referred to standards-based 
curriculum and research-based instructional strategies as key components of high quality 
afterschool programs. To ensure that students are benefiting from the curriculum and 
instruction, periodic student assessments and program evaluations ought to be conducted. 
However, the findings of this study indicated that student assessment was not a common 
practice across even these high performing programs. In general, at the 53 programs, student 
assessments were characterized as informal, and in forms of spot checking, questioning and 
observing students’ understanding. Qualitative findings further revealed that many staff and 
site coordinators were not familiar with assessment and evaluative tools, and did not have a 
clear understanding of their purposes. 

At the afterschool classroom level, periodic assessment should be conducted and 
focused on monitoring student progress and achievement. Professional development can 
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provide opportunities for afterschool staff to review and understand the purposes and 
importance of assessment and evaluations in relation to student outcomes. Reliable and 
simple evaluation and assessment tools need to be developed and used regularly by the 
afterschool classroom and administrative staff. Program staff can also be trained and 
encouraged to conduct self-assessments to understand their own knowledge of the 
curriculum, and their strengths or weaknesses in implementing the curriculum through 
research-based practices. By implementing more formal and periodic student evaluations, 
afterschool programs can better gauge student learning and strategically connect student test 
scores to the curriculum, as well as make programmatic improvements to better meet student 
needs. 

Program organization evaluation. At the program level, program directors and site 
coordinators need to clearly define the purposes of evaluations, so that program staff can 
actively participate in them; for example, assessment information can be used for self- 
improvement in program planning and curriculum development, other internal evaluations 
can be used to guide relationship building, administrative procedures, and resource 
allocation. Through periodic and systematic evaluations, afterschool staff can be made aware 
of their areas of strengths and weaknesses, and make informed decisions based on the 
evaluation findings. For instance, under the topic of staffing, self-evaluation tools can be 
used to understand staff professional development needs, staff utilization of research-based 
activities, and staff knowledge of standards-based curriculum. Using these assessment 
results, program directors can implement changes, allocate resources, and design professional 
development opportunities to further staff expertise in the needed areas. 

In addition, multiple perspectives should be sought when gathering evaluation data. 
Programs need to be encouraged to request parent, student, and community input when 
evaluating program components. The results of these evaluations ought to serve more than 
accountability needs, with the interpretation of the findings also guiding continuous program 
improvement. Consequently, not only will evaluation help programs run smoothly, but they 
can be conducive to positive relationships with parents, students, and community members 
by giving them voices in program planning and improvement. 

Furthermore, state and federal guidelines can be established to support programs in 
their evaluation process. Policymakers can require programs to engage in periodic 
evaluations, document their progress, identify their strengths and weaknesses, and submit a 
plan for change (such as curriculum modifications and adaptations to meet student needs and 
enhance the program outcomes) in addition to the programs’ accountability reports. As part 
of the guidelines, a system for interpretation of program results ought to be in place for 
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programs to structure their findings and reset their goals each year. Administrative staff 
should be given opportunities to be trained in conducting self-assessments of their programs, 
as well how to translate assessment results into program improvement plans. 

State coordinators can also benefit by participating in evaluation professional 
development. This development may provide additional efficacy and understanding about the 
functioning of afterschool programs and create further clarity when the coordinators review 
the programs’ periodic reports. The state coordinators can make use of the findings in the 
reports to monitor and initiate conversations with the programs, building rapport with the 
programs by providing support and resources during the programs’ development as part of 
the collaborative efforts. 

Finally, to initiate and facilitate the process of self evaluation, this study included an 
assessment guide (attached in Appendix B) based on the instruments developed for this 
study. These self-assessment instruments will lead practitioners toward the goal of ongoing 
evaluation and program improvement. Furthermore, this toolkit can also provide knowledge 
and concrete benchmarks to inform stakeholders including practitioners, funding agencies, 
program officers, local education agencies, and policy makers on the progress that the 
programs are making. Stakeholders can make use of the tool to establish standardized 
expectations for afterschool programs and make data-based decisions in setting policies and 
making appropriations. These standards can be used to monitor and evaluate program 
effectiveness and improvement, as well as allow for the sharing of resources and ideas on 
how to best meet these expectations. 

Contributions of the Partnership 

Other than the contributions that this report brings, the Afterschool Partnership has also 
constructed and been maintaining a web site that houses the toolkits for the six content areas 
(math, reading, science, arts, technology, and homework help). These toolkits have been 
widely used by practitioners, and directly address some of the specified practices identified in 
this study. The toolkits on the six content areas are available at the site: 
www.sedl.org/afterschool. The instruments for the self-assessment guide are attached in the 
Appendix. 

Providing Assistance Through Toolkits 

One of the major goals for this 5-year study is to offer strategies, tools, and technical 
assistance to help address two continuing challenges to afterschool programs: (a) ensuring 
that programs offer high quality, research-based academic content utilizing appropriate 
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methods of teaching and learning; and (b) ensuring that programs are able to attract and 
retain students who participate regularly and thus can benefit from these investments. 

Ensuring that Programs Offer High Quality, Researched-based Academic Content 
Utilizing Appropriate Methods of Teaching and Learning 

In assisting afterschool programs to overcome these challenges, the Afterschool 
Partnership synthesized the study findings on effective practices, and developed toolkits for 
the six content areas under study. These toolkits can provide assistance to programs in 
ensuring high quality, standards-based academic content, and in utilizing research-based 
teaching and learning strategies. Through the use of the Afterschool Toolkit, concrete 
strategies are provided to help practitioners build program and staff capacity. The Toolkit 
further offers suggestions, illustrations, and demonstrations on how programs can embed 
academic learning in all aspects of afterschool activities. During the 5 years of this contract, 
the Toolkit has been consistently revised and updated as data and findings from the 
Partnership became available and as the broader research literature evolved. In addition, 
professional development on the use of the Toolkit has been continuously provided in 
national and regional afterschool conferences and workshops. Feedback gathered from these 
professional development events is also used to continuously improve the Toolkit. 

Ensuring that Programs are Able to Attract and Retain Students Who Participate 
Regularly and Thus Can Benefit from these Investments 

With current policies and laws that point to the accountability of Extra Learning 
Opportunities (ELO), the National Partnership’s self-assessment guide (based on the 
instruments developed for this study) provides practitioners with a vital resource for self- 
evaluation. This toolkit includes instruments for programs to examine their curriculum 
content and instructional practices in the six content areas, and offers general guidelines on 
effective program management and administration. It will help programs to make informed 
decisions on developing strategies to attract and retain students. The contribution of the 
Partnership is illustrated in the Figure 14. 
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As illustrated in Figure 14, with the Toolkit, the Afterschool Partnership offers self- 
improvement opportunities for afterschool programs in all essential program components 
including curriculum design, instructional strategies, opportunities to practice (e.g. 
suggestions for project durations, follow-up activities, project extension possibilities), 
periodic assessments, and use of the results of the assessments to fine-tune the program 
goals. In the future this system can be further expanded and enhanced to include professional 
development and instruments on goal setting and provide a set of validated rubrics (perhaps 
also an analytical software to provide simple statistical results for the programs) so that 
education agencies, funding agencies, and policy makers can use the toolkits to establish 
standardized expectations for afterschool programs. At least once a year, afterschool 
programs should take stock of their progress and make plans for refinement. This process 
will empower sites to make grounded decisions and to improve their program performance. 
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Limitations 



Although the findings of this study have implications for afterschool programming and 
future research, a number of limitations should be noted. First, the program participants in 
this study were carefully selected and represented only a small portion of afterschool 
programs in the nation. Thus, the findings on this sample cannot be generalized to the 
afterschool field as a whole. Secondly, because this study sought to conduct extensive site 
visits and intensive data collection on programs that focused on multiple content areas, there 
were time lags between Year 1 and Year 3 of data collection. Coincidently, this was a period 
of tremendous development and growth, as afterschool activities were generating nationwide 
attention in education and political arenas. This drastic growth could have contributed to the 
differences in program content, structure, and practices across programs over the years. 
Audiences should keep this in mind while interpreting results from one year to the next. 
Finally, the purpose of this study was to identify the indicators of high quality afterschool 
programs within the six content areas. Due to the differences in context, a discontinuity of 
the study design was observed which makes the longitudinal comparisons more difficult. 

Conclusion 

The Partnership has successfully accomplished the five major tasks for this project 
aimed at improving the delivery and quality of academic content, teaching, and professional 
development in afterschool programs. Fifty-three high functioning programs representative 
across eight regional divisions of the nation, including rural and urban programs, community- 
based and school district related programs, were identified using rigorous methods. 
Exemplary practices in program organization, program structure, and especially in content 
delivery were studied. The findings were synthesized into the Afterschool Toolkit that was 
made available to programs nationwide via the world- wide-web. Professional development 
was conducted consistently and extensively throughout the nation. 

To conclude the report, the Partnership will provide a “success story” from one of the 
technology programs: 

XXX kids have become leaders in the day school because of their advanced technology 

skills and increased confidence. XXX kids represented the school at the State 

Superintendent’ s Technology Day at the State Capitol this last winter! 

Technology is a part of the environment at this program. The program’s goal is not to 
“teach technology” but to use technology to engage students in their learning and to help the 
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students to become more successful in school. They have certainly aligned and structured 
their resources and achieved their goal! 

Using the Toolkit and self-improvement model (as shown earlier in Figure 14) to self- 
monitor and enhance the program environment, curriculum content, and instructional 
strategies, all programs will have the capacity and potential to achieve their goal. 
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APPENDIX A 



Goal Setting and Curriculum Practices: 
Standards-based Instruction 



Table A 

Percentage of Instructors’ Use of Standards-based Reading Instruction 



Reading contents 


Yes % 


No % 


Print representation of spoken language. 


92 


8 


Recognizing common text features such as headings, key words, charts. 


89 


11 


Demonstrating awareness of sound-symbol relationships. 


88 


12 


Understanding alphabetic principles (that each letter represents a sound). 


89 


11 


Applying decoding to comprehend text (e.g., breaking apart words to understand 
meaning). 


92 


8 


Reading silently or aloud with fluency (smoothly and easily). 


96 


4 


Self-monitoring/self-correcting reading. 


85 


15 


Vocabulary development. 


96 


4 


Identifying literary devices (e.g., simile, metaphor). 


82 


18 


Understanding antonyms/synonyms. 


82 


18 


Using glossaries, table of contents, chapter headings, and indexes to locate information. 


82 


18 


Applying phonetic strategies to make meaning from text. 


91 


9 


Applying decoding to comprehend text. 


91 


9 


Developing pre-reading strategies. 


91 


9 


Understanding textual features. 


82 


18 


Understanding prefixes, suffixes, and affixes. 


85 


15 


Using pictures and context cues to understand meanings of words. 


91 


9 


Identifying homophones and homographs. 


82 


18 


Understanding story components. 


88 


12 


Self-monitoring for comprehension. 


94 


6 


Making inferences using evidence. 


88 


12 


Reading a variety of literary genres. 


89 


11 


Researching topics using a variety of materials. 


74 


27 


Using text features such as lists, indices, headings. 


69 


31 


Identifying/using text organizational structures to gain meaning from text. 


72 


28 


Applying self-correcting strategies to decode text. 


55 


45 


Making predictions/drawing conclusions. 


76 


24 



table continues 
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Table A ( continued ) 



Reading contents 


Yes % 


No % 


Self-monitoring for reading. 


76 


24 


Vocabulary development. 


83 


17 


Identifying figurative and literary devices. 


75 


25 


Analyzing the purpose of different literary texts. 


61 


39 


Understanding literary techniques. 


61 


39 


Developing and investigating research questions. 


61 


39 


Producing book reports or other written projects. 


45 


55 



86 








Table B 



Percentage of Instructors’ Use of Standards-based Math Instruction 



Math contents 


Yes % 


No % 


Concepts of numbers (e.g., whole numbers, ordinal and cardinal numbers, fractions). 


92 


8 


Addition and subtraction of whole numbers. 


100 


0 


Pattern recognition (e.g., through sorting/classification of objects or sounds). 


100 


0 


Basic conventional math symbols (e.g., plus and minus sign, equals sign). 


100 


0 


Basic spatial relationships (e.g., drawing and describing objects). 


100 


0 


Basic concepts of change (e.g., understanding that amount of change can be 
quantified). 


100 


0 


Basic measurement concepts (e.g., length, volume, weight, area, time). 


85 


15 


Understanding/application of fractions, decimals, percentages. 


93 


7 


Multiplication and division. 


94 


6 


Understanding of patterns (using numbers or shapes). 


93 


7 


Using equations to express relationships between numbers. 


100 


0 


Using graphs, tables, or other graphic representations. 


100 


0 


Working with 2- and 3 -dimensional shapes. 


88 


12 


Basic measurement (length, area, weight, volume). 


88 


12 


Designing studies and collecting data. 


88 


12 


Using fractions, decimals, and percentages. 


50 


50 


Using ratios and proportions. 


50 


50 


Using tables or graphs to represent/analyze problems. 


50 


50 


Understanding/applying basic geometric concepts such as angles, side length, 
perimeter, area. 


50 


50 


Formulas for areas of more complex shapes (e.g., triangles, parallelograms, 
trapezoids, circles, pyramids, cylinders). 


50 


50 


Problem solving using equations. 


50 


50 


Designing small research studies. 


100 


0 


Understanding/applying mean, range, and median. 


50 


50 


Representing data in charts, such as histograms, scatter plots, or box plots. 


100 


0 



87 
















Table C 

Percentage of Instructors Use of Standards-based Arts Instruction 



Arts content 


Yes % 


Don’t 
Know % 


Create arts experiences expressing self or environment. 


85 


8 


Apply art techniques and processes. 


93 


0 


Make connections to history and culture. 


67 


15 


Engage students in analyzing and communicating. 


82 


4 


Integrate the arts with other subjects. 


58 


27 


Use technology as a creative tool. 


33 


15 


Listen to, analyze, describe music. 


59 


15 


Sing in groups or individually. 


67 


15 


Play a musical instrument. 


41 


15 


Learn skills in theatre arts performances. 


59 


7 



88 









Table D 



Percentage of Instructors’ Use of Standards-based Science Instruction 



Science content 


Yes % 


Don’t 
Know % 


Understanding concepts of scientific inquiry 


71 


25 


Properties / position / motion of objects. 


60 


38 


Light, heat, electricity, magnetism. 


54 


39 


Characteristics and life cycles of organisms. 


68 


26 


Position and motion of objects. 


57 


38 


Objects and changes in earth and sky. 


56 


33 


Understanding of science/technology. 


80 


18 


Personal health and nutrition. 


77 


16 


Changes in populations and environments. 


58 


36 


Changes in properties in matter. 


66 


30 


Motions and forces. 


65 


29 


Transfer of energy. 


60 


33 


Structure / function of living systems. 


58 


30 


Reproduction and heredity. 


33 


54 


Regulation and behavior. 


48 


44 


Populations and ecosystems. 


49 


35 


Diversity and adaptations of organisms. 


52 


33 


Structure of earth’s system / solar system. 


62 


27 


Populations /resources and environments. 


51 


41 


Natural hazards, risks and benefits. 


50 


44 


History of nature and science. 


46 


40 



89 















Table E 



Percentage of Instructors’ Use of Standards-based Technology Instruction 



Technology contents 


Yes % 


Don’t know % 


Applying strategies to hardware and software. 


60 


13 


Understanding the nature and operation of tech, systems. 


87 


7 


Using tech while working independently. 


81 


13 


Using tech to process data and report results. 


50 


21 


Using tech, to collaborate with peers on projects. 


63 


19 


Using telecom to collaborate with peers and others. 


79 


0 


Using tech, resources for real-world problems and concerns. 


80 


0 


Using tech as a tool for creative projects. 


87 


7 


Using tech to locate, evaluate, and collect info. 


88 


13 


Evaluating and selecting tech tools. 


73 


20 


Practicing responsible behavior and use of tech. 


100 


0 


Discuss consequences of misuse of tech systems. 


86 


7 


Research and evaluate the accuracy and bias of electronic info. 


43 


36 



90 











APPENDIX B 



Goal Setting and Curricular Practices: 
Instructional Strategies 
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Figure A. Frequency of specific instructional practices in math program. 




Including Using reading Using vwiting Using fluency- Having Providing Providing 



nonfictiontext 


centers 


centers 


building 


students 


opportunities 


opportunities 


in struction 






centers 


practice 


for read- 


for studentsto 










independent 


alouds 


link personal 










reading 




experiences 



Addingto a 
"bank" of 
vocabulary 
v\ords 



Figure B. Frequency of specific instructional practices in reading program. 
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Figure C. Frequency of specific instructional practices in arts program. 




Project-based Use of learning Blending science Providing direct Having students Individualized IfUorking lAiththe Providing Letting your Providing 
learning practices centers/ center- practices lAith feedback to work in smaller instruction students as one additional support students know different types of 

based practices practices in other individual groups large group for English your expectations science 

areas students about learner students and criteria instruction to 

progress students 



□ Instructors 



Figure D. Frequency of specific instructional practices in science program. 



92 




technology tools in tools to eual. tools to solve to prom ote higher- cooperative/ connected to day organizations tools available to into all content 

general information problems level thinking collaborative teams school curriculum support afterschool students areas 

prog. 



Figure E. Frequency of specific instructional practices in technology program. 
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Figure F. Percentage of homework instructors engaged in strategies. 
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APPENDIX C 



Evaluative Structures and Program Impact: 
Day-school Teacher Perception of Changes (Figures G-J) 
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Figure G. Teacher perception of changes in student performance and attitude toward arts. 
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Figure FI. Teacher perception of changes in student performance and attitude toward science. 
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Figure /. Teacher perception of changes in student performance and attitude toward technology. 
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Figure J. Teacher perception of changes in student performance and attitude toward homework. 
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Validation Models by Contents 
Technology Validation Model 
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Arts Validation Model 
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Science Validation Model 
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Program Organization and Contextual Variables 
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Work Experience 



Table 1 

Staff Experience at Math Programs 



Years of 


Site Coordinators 


Instructors 


Afterschool Program (ASP) experience 


N= 6 


A =37 


Average years at current site 


4.8 


3.4 


Average years at ASPs in general 


5.0 


3.5 



Table 2 

Staff Experience at Reading Programs 



Years of 


Site Coordinators 


Instructors 


Afterschool Program (ASP) experience 


N= 15 


N= 110 


Average years at current site 


4.1 


3.6 


Average years at ASPs in general 


4.6 


3.8 



As improvement of the study instruments, the scale for reporting work experiences for 
arts, science, technology, and homework were restructured. Percentages were calculated for 
each time periods for better representations. 



Table 3 

Work Experiences of Art Staff 



Length of time working 


In general 


At current site 


Less than a month 


0% 


3% 


A few months 


10% 


31% 


A year 


7% 


0% 


1-3 years 


33% 


44% 


4-7 years 


20% 


19% 


More than 7 years 


30% 


03% 
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Table 4 



Science Work Experience 



Length of time working 


Instructors 


Instructors 


Less than a month 


0% 


0% 


A few months 


12% 


17% 


A year 


6% 


6% 


1-3 years 


38% 


30% 


4-7 years 


36% 


26% 


More than 7 years 


8% 


22% 



Table 5 

Work Experiences of Technology Staff (N = 16) 



Length of time working 


In general 


At current site 


A few months 


13% 


13% 


A year 


6% 


13% 


1-3 years 


19% 


38% 


4-7 years 


50% 


31% 


More than 7 years 


13% 


6% 



Table 6 

Work Experiences of Homework Instructors ( N 


= 50) 




Length of time working 


In general 


At current site 


A few months 


14% 


20% 


A year 


8% 


18% 


1-3 years 


36% 


32% 


4-7 years 


28% 


30% 


More than 7 years 


14% 


0% 
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APPENDIX E 



THE AFTERSCHOOL PROGRAM ASSESSMENT SYSTEM 

Denise Huang, Jamie Cho, and Hannah Nam, Christine Oh, Aletha Harven, and Seth Leon 
CRESST/University of California, Los Angeles 



The National Partnership for Quality Afterschool Learning (the Partnership) assists 21st 
Century Community Learning Centers (CCLC) grantees to build local capacity in academic 
contents to enhance students’ learning. The goal of this assessment system is to provide 
afterschool directors and coordinators with on-site tools for self evaluations, data-based 
decision making, and informed program improvements. 

Indicators for High Functioning Programs 

The findings in the National Partnership Study indicated that the characteristics of the 
identified high functioning afterschool programs (i.e., programs that have demonstrated 
sustained student academic success) appeared to be in alignment with the criteria extracted 
for exemplary afterschool practices in terms of program structure, student engagement, 
instructional strategies, and issues of accountability. From the National Partnership Study, 
the common characteristics that were found across the identified afterschool programs were: 

• These programs had strong leadership and established clear goals 

• They aligned program structures and contents to meet those goals 

• They set a schedule that allowed time for students to leam and practice 

• They established relationships with the day school 

• Their curriculum in general reflected a linkage to standards 

• Most used research-based strategies 

• They all maintained some form of evaluative structures 

• Staff members had low turnover rates 

• Staff members related well to the students 

• Staff members had built rapport, maintained high expectations, and kept students 
motivated and engaged. 

Assessment for Continuous Program Improvement 

Quality afterschool programs are very concerned about the overall effectiveness of their 
programs and make assessment and evaluation an integral part of their functions. Evaluations 
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help programs to discover their strengths and weaknesses and determine areas where changes 
are needed. With careful refinement and constant monitoring of an afterschool plan, 
managers of afterschool programs can judge the efficacy of their efforts based on established, 
accepted goals for the program and improve their effectiveness in promoting student 
outcomes. 

Based on the Partnership Study, self-assessment survey questionnaires are prepared for 
individual program use. Since the main focus of this study is on the instructional features of 
the six content areas (math, reading science, arts, technology, and homework assistance), 
separate instruments are prepared for each individual contents. A more generic and 
condensed kit for self-assessing the program environment and management is provided in 
Exhibit 1, where a rubric for scoring is also included. The intention is to assist afterschool 
programs in making a head start for initiating self-assessment for continuous improvement. 
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Afterschool Program Evaluation 
Staff Self-Assessment Survey 

Instructions 

Eight surveys and corresponding rubrics were created based on the following topics of 
program structure: 

• Program Goals 

• Program Code of Conduct 

• Program Organization 

• Linkages to Day School 

• Curriculum Design 

• Professional Development 

• Evaluation 

• Communication 

When using these self-assessment surveys, programs are encouraged to choose topics 
based on the areas of their greatest needs for program improvement. There is no need to 
complete all eight surveys. 

Managerial staff (e.g. program administrators, project directors, site coordinators, etc.) 
should collect the surveys completed by site-level staff members (e.g., instructors, 
volunteers, tutors, etc.) and use the rubrics to assess current status of their program. 

Program Goals 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. The program has clear established 
goals. 


1 


2 


3 


4 


5 


b. Staff use program goals to plan 
activities. 


1 


2 


3 


4 


5 


c. Activities are geared toward helping 
students achieve program goals. 


1 


2 


3 


4 


5 


d. Students are aware of program goals. 


1 


2 


3 


4 


5 


e. Parents are aware of program goals. 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Scores per 




survey 




5-10 


Program goals are not clearly articulated and inconsistently implemented across afterschool 
programming. The program should make an effort to clearly define goals for each 
programming area. In addition, a strategic plan should be undertaken to ensure that program 
curriculum is aligned with the program goal and that staff have proper training to carry out 
the curriculum. For example, a program with a goal of improving achievement should include 
standards-based content in the curriculum. The program could also consider hiring qualified 
teachers who have familiarity with the content standards. 


11-19 


The articulation or implementation of program goals may be less than ideal for some 
programs. The program could review their goals to identify areas where the goals may need to 
be clarified or communicated more effectively to staff. In addition, the program could also 
review alignment between program goals, curriculum content and staff capabilities to 
determine if certain program areas need improvement. 


20-25 


Program goals are clearly articulated and are being consistently implemented across 
afterschool programming. 



Program Code of Conduct 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. Program has established clear 


participation and attendance 
expectations for students. 


1 


2 


3 


4 


5 


b. Staff set appropriate limits for students. 


1 


2 


3 


4 


5 


c. Staff encourage students to resolve 
their own conflicts. 


1 


2 


3 


4 


5 


d. Staff consistently apply rewards and 
consequences. 


1 


2 


3 


4 


5 


e. Staff are knowledgeable about 


students’ respectable culture(s) and 
language(s). 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 

Score per 
survey 

Clear rules and expectations for conduct need to be established regarding program attendance and 
participation. The site should also ensure that staff have adequate access to professional 
development opportunities geared toward improving general classroom management techniques. 
Staff should be strongly encouraged to participate in these opportunities to improve their capacity 
to deal with student issues such as discipline and conflict resolution. 

11-19 Program rules and expectations for conduct may not be as clear they could be. The site could 
review program rules and expectations to identify areas where they may need to be clarified or 
communicated more effectively to staff. In addition participation in professional development 
training could be monitored to determine whether levels are adequate. 

20-25 There are clear rules and expectations for conduct within the program. 



5-10 
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Program Organization 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. Students understand the rules for 
behavior. 


1 


2 


3 


4 


5 


b. The site addresses student behavioral 
issues in a timely and consistent 
manner. 


1 


2 


3 


4 


5 


c. The staff has a voice in decisions 
about curriculum and instruction. 


1 


2 


3 


4 


5 


d. Staff members take an active role in 
program leadership and decision- 
making. 


1 


2 


3 


4 


5 


e. Staff members meet with each other 
on a regular basis to review data on 
student learning. 


1 


2 


3 


4 


5 


f. Supervisors provide adequate 
support to their staff 


1 


2 


3 


4 


5 


g. An adequate selection of 

texts/materials/resources is available 
for [CONTENT AREA] instruction. 


1 


2 


3 


4 


5 


h. The program is structured in a way 
that encourages me to have positive 
working relationships with the 
students. 


1 


2 


3 


4 


5 


i. I have experience in working with 
diverse student populations. 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per survey 


9-18 


The program structure lacks in organization. The site needs to establish clear 
program rules, regulations, and behavior expectations for students and identify areas 
where they may need to be communicated more effectively to students and staff. 

The program should review the resources available to the site/staff, such as time for 
regular meetings, support for staff, and materials for instruction. Staff should be 
encouraged to partake in the decision-making process. 


19-35 


The system of organization is adequate but needs further work on improving 
program organization and structure for maintaining program efficiency. The 
program could review program rules, regulations, and behavior expectations for 
students to identify areas where they may need to be clarified. The program may 
meet the minimal requirements for resources and instruction, but should identify a 
means to enrich the program, such as regular and meaningful staff meetings or a 
review of current instructional materials/resources to ensure staff and student 
satisfaction. The program could work on improving the sense of autonomy and 
teamwork among the program for staff members to encourage a sense of ownership 
in the decision-making process. 


36-45 


There is a system of organization for the program and for retaining quality staff. 



Linkages to Day School 

Please rate the extent to which you agree or disagree with the following statements about linkages to day school. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. On a week-to-week basis the staff 
know what [CONTENT AREA] 
content will be covered during the 
school day at the students’ schools. 


1 


2 


3 


4 


5 


b. The staff know whom to contact at the 
students’ day schools if they have 
questions about their students’ progress 
or status. 


1 


2 


3 


4 


5 


c. The staff coordinate their afterschool 
activities/projects with the students’ 
day school work. 


1 


2 


3 


4 


5 


d. Day school and afterschool staff 
collaborate on developing curriculum 
and activities. 


1 


2 


3 


4 


5 


e. The staff consider/use assessment data 
from students’ day school to plan their 
work with students. 


1 


2 


3 


4 


5 



ill 









Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 
survey 

5-10 Staff are unaware of the day school curriculum and activities and the students’ progress at school. 
Afterschool staff and day school teachers need to coordinate activities and curriculum 
development and curriculum implementation. Afterschool activities should be aligned with the 
students’ day school performance. Staff need to be encouraged to interact with day school 
teachers in order to learn more about student progress, effective instructional techniques, and 
curricula implementation. Also, staff should learn how to use student assessment information to 
inform their lesson plans and activities. 

11-19 Staff have some contact with day school staff about lesson plans, curriculum development and 
implementation. Afterschool staff may inquire about student progress, but interactions could be 
more frequent. Staff should increase their efforts to coordinate with day school teachers in order 
to understand their student’s needs. Also, efforts could be made to learn about effective teaching 
strategies from student’s teachers. 

20-25 Staff frequently/regularly communicates with day school teachers and parents. 
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Curriculum Design 

Please rate the extent to which you agree or disagree with the following statements about curriculum design. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. The instructional staff are 
knowledgeable about national 
[CONTENT AREA] standards. 


1 


2 


3 


4 


5 


b. The instructional staff are 
knowledgeable about state 
[CONTENT AREA] standards. 


1 


2 


3 


4 


5 


c. The curriculum is based on 
[CONTENT AREA] standards. 


1 


2 


3 


4 


5 


d. Staff regularly adjust/alter afterschool 
practices based on data about student 
learning (e.g. test results, student 
performance/progress, student work). 


1 


2 


3 


4 


5 


e. The curriculum is consistent with 
program objectives. 


1 


2 


3 


4 


5 


f. The language and culture of the 
students are taken into account when 
designing activities. 


1 


2 


3 


4 


5 



Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 



survey 


6-12 


The curriculum design of the program needs more direction. The program needs to review their 
program objectives to ensure clarity and identify areas that need to be communicated effectively 
to staff. The curriculum needs to be reviewed to ensure that it meets state and national based 
standards, and that it is being designed for a diverse-set audience. The site should also ensure that 
staff have adequate access to opportunities (e.g. professional development) geared toward 
learning about state and national standards. Staff should be strongly encouraged to participate in 
these opportunities to improve their capacity to instruct students. 


13-23 


The curriculum minimally meets standards and objectives in its design. The program could have a 
more strategic approach in achieving program objectives through its curriculum by providing 
higher level standards in curriculum instruction or accommodating to the culture and language of 
its students. Staff could be encouraged to participate in professional development opportunities to 
improve their capacity to instruct students. Staff could also use student data (e.g. test results, 
student performance/progress, student work) to improve current practices. 


24-30 


The curriculum is designed to be meaningful, addresses state and national based standards, and 
accommodates a diverse student population. 
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Professional Development 

Please rate the extent to which you agree or disagree with the following statements about professional 
development opportunities offered from your afterschool program. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. Staff are offered multiple opportunities 
for professional developments 
annually. 


1 


2 


3 


4 


5 


b. Opportunities are relevant to staff 
training needs. 


1 


2 


3 


4 


5 


c. Professional developments offered are 
aligned with program goals. 


1 


2 


3 


4 


5 


d. All staff received trainings prior to 
content specific lesson 
implementations. 


1 


2 


3 


4 


5 


e. Staff requests for additional training 
and/or professional development are 
always addressed. 


1 


2 


3 


4 


5 



Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 



survey 


5-10 


Staff do not have adequate training to ensure high quality instruction. The program needs to 
review its current professional development offerings for adequate frequency and relevancy for 
its staff. Offerings need to be aligned with program goals to ensure clear program 
direction/mission. Trainings should be offered multiple times throughout the year, and staff 
should have adequate resources (i.e., time and financial) to participate in professional 
development. In addition, the program should identify professional development needs through 
staff requests. 


11-19 


Staff are provided adequate training. The program could look to improving their professional 
development offerings by reassessing the frequency and relevancy for its staff. Improving the 
offerings may increase staffs ability to develop and implement appropriate curriculum, support 
their function as role models, and generally attract and retain high quality staff. 


20-25 


All staff members have adequate training to ensure high quality instruction. 
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Example of identifying professional development through staff requests: 

Please check the top three areas in which you’d like further training: 

□ Connecting afterschool learning w/ day-school curriculum 

□ Applying state/national standards to afterschool curriculum 

□ Evaluating the program/self evaluation 

□ Working with English language learners (students who do not speak English as 
their first language) 

□ Content- specific training 

□ Assessment 

□ Other: 
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Evaluation 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. Staff are given opportunities to share 
feedback about the program. 


1 


2 


3 


4 


5 


b. Staff are given continuous feedback on 
their performance. 


1 


2 


3 


4 


5 


c. If the afterschool program is evaluated 
(either formally or informally), the 
staff are notified of the findings from 
these evaluations. 


1 


2 


3 


4 


5 


d. Students are given opportunities to 
share their concerns about the program. 


1 


2 


3 


4 


5 


e. Students’ academic progress is 
frequently evaluated. 


1 


2 


3 


4 


5 


f. Parents are given opportunities to 
provide feedback about the program. 


1 


2 


3 


4 


5 


g. Evaluation findings are used for 
program improvement. 


1 


2 


3 


4 


5 



Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected) 



Score per 



survey 


7-14 


Evaluations (of the program and the staff) are not adequately or effectively used to improve the 
program. Staff need to be informed of program evaluation findings and given feedback of their 
own performance. Staff, students and parents need to be given opportunities to provide feedback 
on the program. The program needs to engage in continuous improvement and make assessment 
and evaluation an integral part of their functions. 


15-27 


Evaluations (of the program and the staff) may be minimally used to improve program. The 
program could review their information sharing system to ensure that staff understand the 
evaluation findings and the feedback of their own performance. The program could look to 
providing frequent opportunities for staff, students, and parents to provide feedback on the 
program. 


28-35 


The program has a program improvement system in place that utilizes evaluation findings and 
feedback. 
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Communication 



1. On average, how often during a given semester do you: 







Never 


Rarely 


Sometimes 


Frequently 


Regularly 


a. 


Speak with your students’ day 
school teachers to coordinate 
curricula? 


1 


2 


3 


4 


5 


b. 


Speak with your students’ day 
school teachers specifically 
regarding [CONTENT AREA] 
instruction issues? 


1 


2 


3 


4 


5 


c. 


Speak with your students’ day 
school teachers regarding 
student progress and homework? 


1 


2 


3 


4 


5 


d. 


Discuss student issues (e.g., 
behavior, progress, special 
events, etc.) with the parents of 
your afterschool students? 


1 


2 


3 


4 


5 


e. 


Speak with your students’ 
parents regarding student 
progress and homework? 


1 


2 


3 


4 


5 


2. 


Please rate the extent to which you agree or disagree with the following statements. 






Indicators 


Strongly 

disagree 








Strongly 

agree 


a. 


There are specific staff members who 
are in charge of coordinating 
communication between the 
afterschool program and the students’ 
day school staff. 


1 


2 


3 


4 


5 



b. I have designated times set aside to 

meet with the parents of my afterschool 1 2 3 4 5 

students. 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 
survey 

7-14 The program needs to establish a clear plan for communication with the day school and the 

parents. More effort needs to be made in fostering communication between the afterschool staff 
and day school teachers, and afterschool staff and parents. Staff needs to speak with day school 
teachers to coordinate curricula and discuss instruction issues or student progress and homework. 
Also, staff needs to increase interaction with parents to discuss student issues or student progress 
and homework. Thus, staff should be encouraged to speak more frequently with day school 
teachers in order to acquire knowledge of their students and attain effective teaching strategies to 
help students learn. Also, staff should be encouraged to schedule parent-teacher meetings and find 
ways to increase parent participation so parents will feel connected to the program. 

15-27 The program may have a plan for communication with the day school and the parents, but it could 
be reviewed for improvement or clarity. Staff needs to maintain frequent communication with day 
school teachers and parents. The program could review current efforts in fostering communication 
and assess areas that need improvement. Staff could be encouraged to share more information 
about the afterschool program with parents so that parents are aware of events and activities as 
well as student progress. In addition, staff could schedule frequent meetings with day school 
teachers and parents, and be given the time to schedule meetings with parents and day school 
teachers outside of program hours. 

28-35 The program has a clear plan for communication with the day school and parents. 
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Instructional Features 
Program Staff Self-Assessment Survey 



Instructions 

Three surveys and corresponding rubrics were created based on the following topics of 
program structure in a content- specific program: 

• Program Goals 

• Program Support 

• Program Evaluative Structure 

When using these self-assessment surveys, content-specific programs are encouraged to 
choose topics based on the areas of their greatest needs for program improvement. There is 
no need to complete all three surveys. 

Managerial staff (e.g. program administrators, project directors, site coordinators, etc.) 
should collect the surveys completed by site-level staff members (e.g. instructors, volunteers, 
tutors, etc.) and use the rubrics to assess current status of their program. 



[CONTENT AREA such as Math, Reading, Science, etc.] Program Goals 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. This [CONTENT AREA! program 
has clear established objectives for 
students to achieve. 


1 


2 


3 


4 


5 


b. Staff use these objectives to plan 
activities. 


1 


2 


3 


4 


5 


c. Activities are geared toward helping 
students achieve program objectives. 


1 


2 


3 


4 


5 


d. Students are aware of program 
objectives. 


1 


2 


3 


4 


5 


e. Parents are aware of program 
objectives. 


1 


2 


3 


4 


5 


f. Sufficient time is allocated to the 
[CONTENT AREAj program for 
student practice and activities 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 
survey 


6-12 


Program goals are inconsistently implemented and not clearly articulated across the [CONTENT 
AREA] programming. The [CONTENT AREA] program should make an effort to clearly define 
goals for its content area. In addition, a strategic plan should be undertaken to ensure that 
[CONTENT AREA] curriculum is aligned with the program goals and that staff have proper 
training to carry out the curriculum. For example, a program with the goal of improving 
[CONTENT AREA] achievement should include standards-based [CONTENT AREA] content 
in the curriculum. The program should look to hire qualified teachers who have familiarity with 
the [CONTENT AREA] standards. 


13-23 


The articulation or implementation of [CONTENT AREA] program goals may be less than ideal 
for some programs. The [CONTENT AREA] program should review their goals to identify areas 
where the goals may need to be clarified or communicated more effectively to staff. In addition, 
the [CONTENT] program should also review alignment between its goals, [CONTENT AREA] 
curriculum content and staff capabilities to determine if certain program areas need improvement. 


24-30 


[CONTENT AREA] program goals are clearly articulated and consistently implemented across 
[CONTENT AREA] programming. 


For additional content specific training, please go to the SEDL Afterschool Training Toolkit website 
(http://www.sedl.org/afterschool/toolkits/). 



[CONTENT AREA] Program Support 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. The staff has a voice in decisions about 
curriculum and instruction. 


1 


2 


3 


4 


5 


b. Staff members meet with each other on 
a regular basis to review data on 
student learning. 


1 


2 


3 


4 


5 


c. [CONTENT AREA] instruction is a 
scheduled, integral part of the 
afterschool program. 


1 


2 


3 


4 


5 


d. An adequate selection of 

texts/materials/resources is available 
for [CONTENT AREA] instruction. 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 
survey 


4-8 


The [CONTENT AREA] program structure lacks in support and resources. The program should 
review the resources available to the site/staff, such as time for regular meetings, support for staff, 
and materials for [CONTENT AREA] instruction. Staff should be encouraged to partake in the 
decision-making process. 


9-15 


The system of support is adequate but needs further work. The [CONTENT AREA] program 
may meet the minimal requirements for [CONTENT AREA] resources and instruction, but 
should identify a means to enrich the program, such as regular and meaningful staff meetings or a 
review of current [CONTENT AREA] instructional materials/resources to ensure staff and 
student satisfaction. The program should encourage staff to participate in the decision-making 
process. 


16-20 


There is a system of support and the staff effectively utilizes the [CONTENT AREA] program 
resources. 


For additional content specific training, please go to the SEDL Afterschool Training Toolkit website 
(http://www.sedl.org/afterschool/toolkits/). 



[CONTENT AREA] Program Evaluative Structure 

Please rate the extent to which you agree or disagree with the following statements. 



Indicators 


Strongly 

disagree 








Strongly 

agree 


a. Students’ academic progress is 
frequently examined. 


1 


2 


3 


4 


5 


b. Student assessment results are used for 
program improvement. 


1 


2 


3 


4 


5 


c. I consider/use assessment data from 
students’ day school to plan my work 
with students. 


1 


2 


3 


4 


5 


d. Student feedback is solicited and used 
for program improvement 


1 


2 


3 


4 


5 
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Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 



Score per 
survey 


4-8 


Student feedback and assessment results are not adequately or effectively used to improve the 
[CONTENT AREA] program. Staff should be given the resources to access student academic 
progress and assessment results from the day school in order to engage in continuous 
[CONTENT AREA] program improvement. Also, staff should be properly taught to utilize the 
assessment results/data in [CONTENT AREA] instruction. Student feedback on the program is 
also an integral part of program improvement. A system for obtaining student feedback should be 
established. 


9-15 


Student feedback and assessment results are minimally used to improve the [CONTENT AREA] 
program. The [CONTENT AREA] program should ensure that staff are properly and effectively 
utilizing the student assessment results/data in [CONTENT AREA] instruction. The 
[CONTENT AREA] program should look to providing frequent opportunities for student 
feedback. 


16-20 


The [CONTENT AREA] program has a program improvement system in place that utilizes 
student assessment data/results and feedback. 


For additional content specific training, please go to the SEDL Afterschool Training Toolkit website 
(http://www.sedl.org/afterschool/toolkits/). 
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Program Staff Informational Sheet 
Instructional Features: Standard and Research-based Practices: 

Math 

The following list illustrates recommended mathematical practices that should be used 
regularly as part of your afterschool program. Your program does not need to cover all of the 
recommendations. Rather, this list serves as a tool to help you better understand the focus of 
your program goals and align activities and instructional strategies accordingly. 

• Let students know your expectations and grading criteria for the afterschool math 
assignments. 

• Provide direct feedback to individual students about their progress. 

• Provide individualized instruction, such as one-on-one tutoring, peer support, or 
computer-assisted instruction. 

• Provide additional support for students who do not speak English as their first 
language. 

• Provide specific instruction on how to solve math problems. 

• Use mathematical tools, such as manipulatives, calculators, and computer-based 
tools. 

• Provide unstructured opportunities for students to investigate and explore their 
ideas about math on their own. 

• Have students actively participate in hypothesis testing. 

• Have students provide written justification for their work. 

• Ask students to solve “real world” problems. 

• Connect math instruction to day school curriculum. 

• Blend math practices with practices in other content areas, such as social studies, 
reading, or art. 

• Use learning centers and center-based practices. 

• Use project-based learning practices (e.g., having students work on projects 
spanning several days). 

• Use service learning practices (e.g., linking instruction to community service). 

• Community partnerships with math-related organizations help support the 
afterschool program. 



123 




Recommended Math Concepts for Grades K-2 

The following list shows standards-based mathematical instructions that should be used 
for students in Grades K-2 in your afterschool program. Your program does not need to 
cover all of the recommended concepts. Rather, the list serves as a tool to help you better 
align activities and instructional strategies with age- and grade- appropriate mathematical 
skills. 

• Concepts of numbers (e.g., whole numbers, ordinal and cardinal numbers, 
fractions). 

• Addition and subtraction of whole numbers. 

• Basic conventional math symbols (e.g., plus and minus sign, equals sign). 

• Basic concepts of change (e.g., understanding that amount of change can be 
quantified). 

• Basic measurement concepts (e.g., length, volume, weight, area, time). 

• Basic spatial relationships (e.g., drawing and describing objects). 

• Pattern recognition (e.g., through sorting/classification of objects or sounds). 

Recommended Math Concepts for Grades 3-5/6 

The following list shows standards-based mathematical instructions that should be used 
for students in Grades 3-5/6 in your afterschool program. Your program does not need to 
cover all of the recommended concepts. Rather, the list serves as a tool to help you better 
align activities and instructional strategies with age- and grade-appropriate mathematical 
skills. 

• Understanding/application of fractions, decimals, percentages. 

• Multiplication and division. 

• Using equations to express relationships between numbers. 

• Using graphs, tables, or other graphic representations. 

• Understanding of patterns (using numbers or shapes). 

• Working with 2- and 3-dimensional shapes. 

• Basic measurement (length, area, weight, volume). 

• Designing studies and collecting data. 

Recommended Math Concepts for Grades 6-8/9 

The following list shows standards-based mathematical instructions that should be used 
for students in Grades 6-8/9 in your afterschool program. Your program does not need to 
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cover all of the recommended concepts. Rather, the list serves as a tool to help you better 
align activities and instructional strategies with age- and grade- appropriate mathematical 
skills. 

• Using fractions, decimals, and percentages. 

• Using ratios and proportions. 

• Understanding/applying mean, range, and median. 

• Using tables or graphs to represent/analyze problems. 

• Representing data in charts, such as histograms, scatter plots, or box plots. 

• Understanding/applying basic geometric concepts such as angles, side length, 
perimeter, area. 

• Formulas for areas of more complex shapes (e.g., triangles, parallelograms, 
trapezoids, circles, pyramids, cylinders). 

• Problem solving using equations. 

• Designing small research studies. 
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Program Staff Informational Sheet 
Instructional Features: Standard and Research-based Practices 

Reading 

The following list illustrates recommended reading practices that should be used 
regularly as part of your afterschool program. Your program does not need to cover all of the 
recommendations. Rather, this list serves as a tool to help you better understand the focus of 
your program goals and align activities and instructional strategies accordingly. 

• Let your students know your expectations and criteria for their afterschool 
assignments. 

• Provide direct feedback to individual students about their progress. 

• Provide individualized instruction (e.g., one-on-one tutoring, peer support, 
computer-assisted instruction). 

• Provide opportunities for independent reading (e.g., computer or tape assisted). 

• Provide opportunities for read-alouds. 

• Include nonfiction text in instruction. 

• Provide opportunities for students to link personal experiences with stories. 

• Connect reading instruction to day school curriculum. 

• Blend reading practices with practices in other content areas (such as social 
studies, math, art, etc). 

• Add to a “bank” of vocabulary words. 

• Use reading centers. 

• Use writing centers. 

• Use fluency-building centers. 

• Provide additional support for students who do not speak English as their first 
language. 

• Use project-based learning practices (having students work on projects spanning 
several days). 

• Use service learning practices (linking instruction to community service). 

• Community partnerships with reading-related organizations helping support the 
afterschool program. 

Reading Staff Self-Assessment (Grades K-2) 

The following list shows standards-based reading instructions that should be used for 
students in grades K-2 in your afterschool program. Your program does not need to cover all 
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of the recommended contents. Rather, the list serves as a tool to help you better align 
activities and instructional strategies with age- and grade-appropriate reading skills. 

• Print representation of spoken language (i.e., understanding that print represents 
spoken words). 

• Demonstrating awareness of sound-symbol relationships. 

• Understanding alphabetic principles (that each letter represents a sound). 

• Recognizing common text features such as headings, key words, illustrations, 
maps, charts. 

• Applying decoding to comprehend text (e.g., breaking apart words to understand 
meaning). 

• Vocabulary development. 

• Reading silently or aloud with fluency (smoothly and easily). 

• Self-monitoring/self-correcting reading. 

• Identifying literary devices (e.g., simile, metaphor). 

• Understanding antonyms/synonyms. 

Reading Staff Self-Assessment (Grades 3-5/6) 

The following list shows standards-based reading instructions that should be used for 
students in Grades 3-5/6 in your afterschool program. Your program does not need to cover 
all of the recommended contents. Rather, the list serves as a tool to help you better align 
activities and instructional strategies with age- and grade-appropriate reading skills. 

• Developing pre-reading strategies. 

• Applying phonetic strategies (i.e., breaking words into sounds) to make meaning 
from text. 

• Applying decoding to comprehend text (e.g., breaking apart words to understand 
meaning). 

• Understanding prefixes, suffixes, and affixes. 

• Identifying homophones and homographs. 

• Understanding textual features (e.g., tables, graphs, lists). 

• Using pictures and context cues to understand meanings of words. 

• Using glossaries, table of contents, chapter headings, and indexes to locate 
information. 

• Understanding story components (such as setting, characters, plot). 

• Self-monitoring for comprehension. 
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• Making inferences using evidence. 

• Reading a variety of literary genres (e.g., short stories, fiction, nonfiction, 
mythology). 

• Researching topics using a variety of materials. 

Reading Staff Self-Assessment (Grades 6-8/9 Middle School) 

The following list shows standards-based reading instructions that should be used for 
students in Grades 6-8/9 in your afterschool program. Your program does not need to cover 
all of the recommended contents. Rather, the list serves as a tool to help you better align 
activities and instructional strategies with age- and grade-appropriate reading skills. 

• Using text features such as lists, indices, headings. 

• Identifying/using text organizational structures (e.g., arrangement, order) to gain 
meaning from text. 

• Applying self-correcting strategies to decode text. 

• Self-monitoring for reading. 

• Vocabulary development 

• Analyzing the purpose of different literary texts. 

• Understanding literary techniques (such as foreshadowing). 

• Identifying figurative and literary devices (e.g., metaphor, simile). 

• Developing and investigating research questions. 

• Making predictions/drawing conclusions. 

• Producing book reports or other written projects. 
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Program Staff Informational Sheet 

Instructional Features-Standard and Research-based Practices 

Science 

The following list illustrates recommended science practices that should be used 
regularly as part of your afterschool program. Your program does not need to cover all of the 
recommendations. Rather, this list serves as a tool to help you better understand the focus of 
your program goals and align activities and instructional strategies accordingly. 

• Let your students know your expectations and criteria for their afterschool 
assignments. 

• Provide direct feedback to individual students about progress. 

• Provide individualized instruction (e.g., one-on-one tutoring, peer support, 
computer-assisted instruction). 

• Provide different types of science instruction to students based on their ability 
level. 

• Provide additional support for students who do not speak English as their first 
language (English learners). 

• Have students work in smaller groups or teams. 

• Provide opportunities for students to conduct simple experiments. 

• Provide opportunities for students to describe scientific procedures. 

• Provide opportunities for students to use tools to gather/analyze/interpret data. 

• Provide opportunities for students to use computer simulations of science 
experiments. 

• Provide opportunities for students to design a scientific investigation. 

• Provide opportunities for students to develop predictions/explanations/models 
using evidence. 

• Use or project-based learning practices (having students work on projects 
spanning several days). 

• Use of learning centers/center-based practices. 

• Connection of science instruction to day school curriculum. 

• Blend science practices with practices in other content areas (such as social 
studies, math, and art). 

• Community partnerships with science-related organizations helping support the 
afterschool program. 

The following list shows standards-based science concepts that should be addressed to 
the students in your afterschool program. Your program does not need to cover all of the 
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recommended concepts. Rather, the list serves as a tool to help you better align activities and 
instructional strategies with age- and grade- appropriate science skills. 

• Understanding of science/technology. 

• Understanding of concepts of scientific inquiry. 

• Properties, position, and motion of objects and materials. 

• Properties/changes of properties in matter. 

• Position and motion of objects. 

• Motions and forces. 

• Transfer of energy. 

• Light, heat, electricity, and magnetism. 

• Populations and ecosystems. 

• Populations, resources, and environments. 

• Characteristics/changes in populations and environments. 

• Characteristics and lifecycles of organisms and environments. 

• Diversity and adaptations of organisms. 

• Structure/function of living systems. 

• Reproduction and heredity. 

• Regulation and behavior. 

• Personal health and nutrition. 

• Structure of the earth’s system/the earth in the solar system. 

• Objects and changes in earth and sky. 

• Natural hazards, risks, and benefits. 

• History of nature and science. 
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Program Staff Informational Sheet 

Instructional Features-Standard and Research-based Practices 

Arts 

The following list illustrates recommended art instructional practices that should be 
used regularly as part of your afterschool program. Your program does not need to cover all 
of the recommendations. Rather, this list serves as a tool to help you better understand the 
focus of your program goals and align activities and instructional strategies accordingly. 

• An arts process of creating, performing, and reflecting (e.g., doing, demonstrating 
learning, group or personal critique). 

• Hands-on activities, performances, or productions. 

• Public performances and presentations as demonstrations of learning (e.g., in- 
house group or community). 

• Opportunities for reflection/inquiry surrounding art activities (e.g., 
reading/writing about the arts). 

• Time for sequential, sustained skill building experiences (e.g., time for students to 
think, practice, explore, make mistakes, and evolve in their understanding). 

• Hands-on work with real world connections (e.g., playing an instrument, creating 
a sculpture, working through a script while incorporating topics that are part of 
the real world environment). 

• Cooperative/collaborative art activities. 

• External collaborations to broaden and deepen experience (e.g., with peers, artists, 
art organizations, community partners). 

• Regular visits/participation by professional artists in afterschool program. 

• Connection of arts instruction to day school curriculum. 

• Field trips to local art museums/concerts/performances. 

• Community partnerships with art-related organizations (beyond field 
trips/performances). 

The following list shows examples of art activities that can be implemented to the 
students in your afterschool program. Your program does not need to cover all of the 
recommended activities. Rather, the list serves as a tool to help you better align activities and 
instructional strategies with age- and grade- appropriate art skills. 

• Creating art experiences that express the self or environment (i.e., activities that 
directly address concerns of students, their culture, and their community). 

• Making connections to history and culture through the arts (i.e., activities that 
focus on understanding the social, historical, or cultural importance of a work of 
art). 
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• Applying art techniques and processes in creating and communicating meaning 
(i.e., activities that build skills, enhance performance, and provide reflection or 
critique of student-developed work). 

• Engaging students in analyzing and communicating about works of art (i.e., 
activities that require talking or writing about a work of art in a way that reflects 
constructs specific to the medium). 

• Integrating the arts with other subjects, either as an instructional strategy or a 
project-based activity (e.g., designing a “travel brochure” for a visit to a favorite 
planet, which integrates art, science, and technology). 

• Using technology as a creative tool (e.g., using computers for animation, graphic 
design, web portfolios). 

• Listening to, analyzing, and describing music. 

• Singing in groups or individually. 

• Playing a musical instrument, in groups or individually. 

• Learning skills in theater arts performance (e.g., script writing, acting, 
improvisation). 
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Program Staff Informational Sheet 

Instructional Features-Standard and Research-based Practices 

Technology 

The following list illustrates recommended practices in technology that should be used 
regularly as part of your afterschool program. Your program does not need to cover all of the 
recommendations. Rather, this list serves as a tool to help you better understand the focus of 
your program goals and align activities and instructional strategies accordingly. 

• Access to technology tools and instruction available to students with different 
levels of knowledge, ability and understanding. 

• Students use multiple technologies as a way to promote inquiry and higher level 
thinking to increase their knowledge and understanding. 

• Students use technology tools to locate, acquire, and evaluate information. 

• Students use technology tools to solve problems, create projects, and 
communicate with others. 

• Students use technology tools for personal, cultural, or real-world issues or 
experiences. 

• Students work in cooperative/collaborative teams or groups while using 
technology for their projects or products. 

• Technology projects and activities connected to day school curriculum. 

• Technology embedded into and across all content areas (e.g., math, science, 
literacy, arts). 

• Community partnerships with technology-related organizations helping support 
the afterschool program. 

Technology Staff Self-Assessment (Grades K-2) 

The following list shows standards-based technology instructions that should be used 
for students in grades K-2 in your afterschool program. Your program does not need to cover 
all of the recommended contents. Rather, the list serves as a tool to help you better align 
activities and instructional strategies with age- and grade-appropriate technology skills. 

• Using a variety of technologies such as computers, printers, scanners, digital 
cameras, and other technology tools appropriate to students’ skills, age, and grade 
level. 

• Using age- and grade-appropriate resources such as interactive books, puzzles, 
logical thinking programs, and multimedia encyclopedias to support learning. 

• Using a variety of technology resources such as writing and drawing tools, digital 
graphics, and other electronic materials for creation and illustrations of thoughts, 
ideas, and stories. 
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• Using electronic resources such as CD-ROM and DVD to gather information for a 
variety of projects. 

• Using technology while working cooperatively or collaboratively with peers, 
family members, and others on authentic or real-world problems. 

• Using electronic tools such as e-mail or the internet to communicate with peers, 
family members, or others regarding authentic or real-world problems or projects. 

• Practicing appropriate and responsible use of computer tools and resources by 
following classroom guidelines. 

Technology Staff Self-Assessment (Grades 3-5/6) 

The following list shows standards-based technology instructions that should be used 
for students in Grades 3-5/6 in your afterschool program. Your program does not need to 
cover all of the recommended contents. Rather, the list serves as a tool to help you better 
align activities and instructional strategies with age- and grade- appropriate technology skills. 

• Using a variety of technologies such as computers, printers, scanners, digital 
cameras, and other technology tools appropriate to students’ skills, age, and grade 
level. 

• Using age- and grade- appropriate resources such as interactive books, puzzles, 
logical thinking programs, and multimedia encyclopedias to support learning. 

• Using a variety of technology resources such as writing and drawing tools, digital 
graphics, and other electronic materials for creation and illustration of thoughts, 
ideas, and stories. 

• Using electronic resources such as CD-ROM and DVD to gather information for a 
variety of projects. 

• Using technology tools such as scientific probes, data collection, and analysis 
applications to gather, assess, or analyze information for a variety of projects. 

• Allowing students to make their own choice of technology tools (from a variety of 
options), in order to carry out individual or collaborative projects or products 
related to academic content. 

• Using technology tools for self-directed, individual project-based activities related 
to academic content. 

• Using technology tools for collaborative, project-based activities related to 
academic content. 

• Using telecommunication tools such as e-mail or the internet to communicate 
with peers, family members, or others regarding authentic or real-world issues or 
projects. 

• Practicing appropriate and responsible use of computer tools and resources by 
following classroom guidelines. 
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Technology Staff Self-Assessment (Grades 6-8/9 Middle School) 



The following list shows standards-based technology instructions that should be used 
for students in Grades 6-8/9 in your afterschool program. Your program does not need to 
cover all of the recommended contents. Rather, the list serves as a tool to help you better 
align activities and instructional strategies with age- and grade- appropriate technology skills. 

• Demonstrating an understanding of the nature and operation of technology 
systems. 

• Applying strategies for identifying and solving common hardware and software 
problems. 

• Using technology while working independently on projects or products related to 
academic curriculum. 

• Using technology to collaborate with peers and others on projects related to the 
academic curriculum. 

• Using telecommunications such as e-mail and the Web to collaborate, publish, 
and interact with peers, experts, and other audiences. 

• Evaluating and selecting technological tools based on the appropriateness for 
specific tasks. 

• Using appropriate technology resources and tools to investigate and solve 
problems and make informed decisions regarding real-world concerns (e.g., using 
technology tools such as scientific probes, data collection, and analysis 
applications to gather, assess, and analyze information for a variety of projects). 

• Using appropriate technologies to locate, evaluate, and collect information from a 
variety of sources. 

• Using technology tools to process data and report results. 

• Researching and evaluating the accuracy, relevance, appropriateness, 
comprehensiveness, and bias of electronic information. 

• Using technology as a tool for creative projects related to writing, publishing, 
music, performance and visual arts, and any other illustration of thoughts and 
ideas. 

• Practicing responsible behavior and use of technology (e.g., using information, 
media, and technology in a responsible manner which includes following the 
school’s acceptable use policy, adhering to copyright laws, respecting the rights 
of others, and employing proper etiquette in all forms of communication). 

• Discussing consequences of misuse of technology systems, information, 
hardware, and software. 
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Program Staff Informational Sheet 
Instructional Features: Standard and Research-based Practices 

Homework 



The following list illustrates recommended homework practices that should be used 
regularly as part of your afterschool program. Your program does not need to cover all of the 
recommendations. Rather, this list serves as a tool to help you better understand the focus of 
your program goals and align activities and instructional strategies accordingly. 

• Let your students know your expectations and criteria for their afterschool 
assignments. 

• Monitor individual student progress. 

• Provide direct feedback to individual students about progress. 

• Incorporation of parents’ input on students’ homework responsibilities and 
progress. 

• Coordination of students’ homework plans with their day school teachers and 
parents. 

• Regular feedback to parents regarding their children’s homework and progress. 

• Provide different types of instruction to students based on their ability level. 

• Individualized instruction (e.g., one-on-one tutoring, peer support, computer- 
assisted instruction). 

• Tutoring component as part of the homework assistance program (i.e., tutors 
trained in & providing specialized help in specific content areas). 

• Provide additional support for students who do not speak English as their first 
language (English learners). 

• Have students work in smaller groups or teams. 

• Instruction on time management or organizational skills. 

• Instruction on test preparation techniques. 

• Instruction on note-taking techniques. 

• Instruction on using reference materials for research. 

• Community involvement in the homework assistance program (e.g., partnerships 
w/ local agencies, universities, volunteer programs). 

The following list illustrates recommended homework strategies to use to monitor 
individual student progress. These strategies should be used regularly as part of your 
afterschool program, but your program does not need to cover all of the recommendations. 
Rather, this list serves as a tool to help you better understand the focus of your program goals 
and align activities and instructional strategies accordingly. 
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• Homework planners/logs 

• Using checklists and rubrics to monitor homework quality and completion 

• Homework hotlines 

• Keeping track of student test scores and progress in the day school 

• Parent-teacher- student learning contracts 

• Graphic organizers 

• Concept maps 
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Parent Satisfaction and Perceived Student Outcomes 
Parent Survey 



Instructions 

For the items below, please circle the number that best reflects your response to each 
statement. 



1. Please rate the extent to which you agree or disagree with the following statements. 





Strongly 

disagree 








Strongly 

agree 


a. I feel welcomed to visit my child(ren)’s 
afterschool program any time I want. 


1 


2 


3 


4 


5 


b. Someone is available to talk to me at 
the afterschool program when I have 
any questions or concerns. 


1 


2 


3 


4 


5 


c. I feel comfortable about asking the 
people who work at the afterschool 
program what my child(ren) is learning 
there. 


1 


2 


3 


4 


5 


d. The afterschool program staff help me 
understand the report cards and paper 
work I get from my child(ren)’s school. 


1 


2 


3 


4 


5 


e. The afterschool program staff will let 
me know immediately if my child(ren) 
is not paying attention to his/her 
schoolwork. 


1 


2 


3 


4 


5 


f. The afterschool program staff deals 
with my child(ren)’s behavior 
problems quickly and fairly. 


1 


2 


3 


4 


5 


2. How satisfied are you with the following parts of your child(ren)’s 


afterschool program? 






Very 

Unsatisfied 








Very 

Satisfied 


a. The kinds of activities offered. 


1 


2 


3 


4 


5 


b. The overall performance of the 
afterschool staff. 


1 


2 


3 


4 


5 


c. What my child(ren) learns in the 
afterschool program. 


1 


2 


3 


4 


5 


d. The materials and resources that the 
program provides for parents 
(workshops, newsletters, parents’ 
nights). 


1 


2 


3 


4 


5 
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3. Since your child(ren) started attending the [CONTENT AREA] activities in the afterschool program, 
have you noticed a change in his/her: 





Not sure 


Decreased 


No 

change 


Improved 
a little 


Improved 

somewhat 


Improved 
a lot 


a. [CONTENT AREA] 

skills. 


0 


1 


2 


3 


4 


5 


b. Interest in [CONTENT 
AREA], 


0 


1 


2 


3 


4 


5 


c. Interest in his/her 


0 


1 






A 


C 


schoolwork in general. 


A- 


D 




J 


d. Finishing of 














[CONTENT AREA] 

homework on time. 


0 


1 


2 


3 


4 


5 



Rubric 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected) 



Score per 



survey 


14-28 


Parents are generally unsatisfied and have low perceptions of the afterschool program. In 
addition, parents perceived the program to have no or negative impact on their child(ren). Parent 
feedback on the program is an integral part of continuous program improvement. A system for 
obtaining parent feedback ought to be established. The feedback should be used to examine and 
identify areas for program improvement. 


29-55 


Parents are minimally satisfied and have average perceptions of the afterschool program. In 
addition, parents perceived the program to have little positive impact on their child(ren). The 
program should solicit additional feedback from parents and provide frequent opportunities for 
feedback. The feedback should be used to examine and identify areas for program improvement. 


56-70 


Parents are generally satisfied and have high perceptions of the afterschool program. In addition, 
parents perceived the program to have positive impact on their child(ren). 
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Teacher Awareness of Afterschool Activities & Perceived Student Outcomes 

Teacher Survey 

Instructions 

For the items below, please circle the number that best reflects your response to each 
statement. 

1. Are you aware of which of your students participate in this afterschool program? 



Yes 


No 


1 


2 



If yes, continue with question #2. If no, please skip to question #4. 



2. Please estimate how the behavior of your students who attend the afterschool program changed in the 
following areas since participating in the program: 





Don’t know/ 
not sure 


Decreased 

greatly 


Decreased 


Stayed the 
same 


Increased 


Increased 

greatly 


a. School attendance. 


0 


1 


2 


3 


4 


5 


b. Frequency of 
classroom 


0 


1 


2 


3 


4 


5 


participation. 














c. Effort on 


0 


1 


9 


'X 


A 




schoolwork. 


Z. 






D 


d. Paying attention in 
class. 


0 


1 


2 


3 


4 


5 


e. Fewer discipline 
problems. 


0 


1 


2 


3 


4 


5 



143 







3. Please estimate how the behavior of your students who attend the afterschool program changed in the 
following areas since participating in the program: 





Don’t know/ Decreased 

not sure greatly 


Decreased 


Stayed 
the same 


Increased 


Increased 

greatly 


a. Quality of their 

[CONTENT AREA] 

homework. 


0 


1 


2 


3 


4 


5 


b. Turning in their 
[CONTENT AREA] 

homework on time. 


0 


1 


2 


3 


4 


5 


c. Their [CONTENT 
AREA] skills. 


0 


1 


2 


3 


4 


5 


d. Their grades on 

[CONTENT AREA] 

tests. 


0 


1 


2 


3 


4 


5 


e. Their attitude toward 

[CONTENT 

AREA], 


0 


1 


2 


3 


4 


5 


f. Cooperative and 
collaborative skills. 


0 


1 


2 


3 


4 


5 


g. Persistence in school 
work. 


0 


1 


2 


3 


4 


5 


h. Problem solving 
skills. 


0 


1 


2 


3 


4 


5 


4. Please rate the extent to which you agree or disagree with the following statements: 






Strongly 

disagree 








Strongly 

agree 


a. I am knowledgeable about the 

[CONTENT AREA] curriculum of the 
afterschool program. 


1 


2 


3 


4 


5 


b. I am knowledgeable about the non- 
academic activities offered through the 
afterschool program. 


1 


2 


3 


4 


5 


c. I know whom to contact at the 
afterschool program if I have a 
question. 


1 


2 


3 


4 


5 
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5. Please estimate how often: 





Never 


Rarely 


Sometimes 


Frequently 


Regularly 


a. The staff of the afterschool 
program make an effort to 
communicate with you about 
your students’ progress. 


1 


2 


3 


4 


5 


b. The staff of the afterschool 
program have contacted you to 
help coordinate curriculum 
between the program and the 
school. 


1 


2 


3 


4 


5 


c. You review the instructional 
materials used in the afterschool 
program. 


1 


2 


3 


4 


5 


d. Your school staff meet with the 
after- school program staff to 
align curricula. 


1 


2 


3 


4 


5 



Rubric - Question 2 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected) 



Score per 



survey 


13-38 


Day school teachers generally perceive the program to have no or negative impact on their 
students’ behavior. A system for obtaining day school feedback needs to be established. Day 
school teacher feedback on the program is an integral part of program improvement. The 
feedback should be used to examine and identify areas for program improvement. Strategic 
collaboration with day school teachers would address the areas that need improvement. 


39-51 


Day school teachers generally perceive the program to have little positive impact on their 
students’ behavior. Day school teacher feedback on the program is an integral part of program 
improvement. The program should solicit additional feedback from day school teachers and 
provide frequent opportunities for feedback. The feedback should be used to examine and identify 
areas for program improvement. Strategic collaboration with day school teachers would address 
the areas that need improvement. 


52-65 


Day school teachers generally perceive the program to have positive impact on their students’ 
behavior. 
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Rubric - Question s 3 & 4 

(To obtain the total score, multiply the scores in the rubric with the total number of surveys collected ) 

Score per 
survey 

7-14 The program needs to establish a clear plan for communication and collaboration with the day 
school. More effort needs to be made in fostering communication and collaboration between the 
afterschool staff and day school teachers. Staff needs to speak with day school teachers to 
coordinate curricula and discuss instruction issues or student progress and homework. Also, staff 
needs to increase interaction with day school teachers to discuss student issues or student progress 
and homework. Staff should be encouraged to schedule meetings with day school teachers. 

15-27 The program may have a plan for communication and collaboration with the day school, but it 
should be reviewed for improvement or clarity. Staff needs to maintain frequent communication 
with day school teachers. The program should review current efforts in fostering communication 
and collaboration, and assess areas that need improvement. Staff should be encouraged to share 
more information about the afterschool program with day school teachers so that they are aware 
of events and activities as well as student progress. In addition, staff should schedule frequent 
meetings with day school teachers, and be given the time to schedule meetings with day school 
teachers outside of program hours. 

28-35 The program has a clear plan for communication and collaboration with the day school teachers. 
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